Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> send the whole logs


+
Martinus m 2013-10-08, 10:45
+
David Sinclair 2013-10-08, 13:01
+
Martinus m 2013-10-09, 02:35
+
Hari Shreedharan 2013-10-09, 05:45
+
Martinus m 2013-10-09, 06:26
Copy link to this message
-
Re: send the whole logs
text does not write the headers, try HEADER_AND_TEXT  
Thanks,
Hari
On Tuesday, October 8, 2013 at 11:26 PM, Martinus m wrote:

> Hi Hari,
>
> I tried to add below serializers in my flume.conf :
>
> agent.sinks.s3Sink.serializer = text
>
> And it's still doesn't have timestamp (date) info from the original log message :
>
> Thanks,
>
> Martinus
>
>
>
> On Wed, Oct 9, 2013 at 1:45 PM, Hari Shreedharan <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > The timestamp is in the event header. You would need to use a serializer which also writes out the headers.
> >
> >
> > Thanks,
> > Hari
> >
> >
> > On Tuesday, October 8, 2013 at 7:35 PM, Martinus m wrote:
> >
> > > Hi David,
> > >
> > > I'm using Syslog UDP source and on the syslog messages it have it's timestamp for each message. I also use HDFS sink, but when I saw the result message on HDFS folder, it doesn't have the timestamp (date) info.
> > >
> > > Thanks,
> > >
> > > Martinus
> > >
> > >
> > > On Tue, Oct 8, 2013 at 9:01 PM, David Sinclair <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > > > Martinus,
> > > >
> > > > Can you give a little more details? It sounds like you want to use the Spooling Directory Source, http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source, but if you can be clearer about your requirements, I may be able to help you better.
> > > >
> > > > dave
> > > >
> > > >
> > > > On Tue, Oct 8, 2013 at 6:45 AM, Martinus m <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > > > > Hi,
> > > > >
> > > > > What configurations should I put on flume.conf to get the whole logs message to be put into the sink?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > Martinus
> > >
> >
>

+
Martinus m 2013-10-09, 07:52
+
David Sinclair 2013-10-09, 12:53
+
Martinus m 2013-10-11, 07:36
+
David Sinclair 2013-10-11, 14:29
+
Jeff Lord 2013-10-12, 06:05
+
Martinus m 2013-10-15, 02:00
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB