Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - send the whole logs


Copy link to this message
-
Re: send the whole logs
Martinus m 2013-10-09, 06:26
Hi Hari,

I tried to add below serializers in my flume.conf :

agent.sinks.s3Sink.serializer = text

And it's still doesn't have timestamp (date) info from the original log
message :

Thanks,

Martinus

On Wed, Oct 9, 2013 at 1:45 PM, Hari Shreedharan
<[EMAIL PROTECTED]>wrote:

>  The timestamp is in the event header. You would need to use a serializer
> which also writes out the headers.
>
>
> Thanks,
> Hari
>
> On Tuesday, October 8, 2013 at 7:35 PM, Martinus m wrote:
>
> Hi David,
>
> I'm using Syslog UDP source and on the syslog messages it have it's
> timestamp for each message. I also use HDFS sink, but when I saw the result
> message on HDFS folder, it doesn't have the timestamp (date) info.
>
> Thanks,
>
> Martinus
>
>
> On Tue, Oct 8, 2013 at 9:01 PM, David Sinclair <
> [EMAIL PROTECTED]> wrote:
>
> Martinus,
>
> Can you give a little more details? It sounds like you want to use the
> Spooling Directory Source,
> http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source,
> but if you can be clearer about your requirements, I may be able to help
> you better.
>
> dave
>
>
> On Tue, Oct 8, 2013 at 6:45 AM, Martinus m <[EMAIL PROTECTED]> wrote:
>
> Hi,
>
> What configurations should I put on flume.conf to get the whole logs
> message to be put into the sink?
>
> Thanks.
>
> Martinus
>
>
>
>
>