Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> send the whole logs


Copy link to this message
-
Re: send the whole logs
Hi Hari,

Thanks, it's worked, but it's timestamp information doesn't looks like the
original one :

{timestamp=1381304766000, host=flume, Severity=6, Facility=3}

The original one is like below :

Oct  9 07:46:06 flume

Is there any other configuration that I miss to make this header just looks
like the original message?

Thanks,

Martinus
On Wed, Oct 9, 2013 at 3:29 PM, Hari Shreedharan
<[EMAIL PROTECTED]>wrote:

>  text does not write the headers, try HEADER_AND_TEXT
>
>
> Thanks,
> Hari
>
> On Tuesday, October 8, 2013 at 11:26 PM, Martinus m wrote:
>
> Hi Hari,
>
> I tried to add below serializers in my flume.conf :
>
> agent.sinks.s3Sink.serializer = text
>
> And it's still doesn't have timestamp (date) info from the original log
> message :
>
> Thanks,
>
> Martinus
>
>
>
> On Wed, Oct 9, 2013 at 1:45 PM, Hari Shreedharan <
> [EMAIL PROTECTED]> wrote:
>
>  The timestamp is in the event header. You would need to use a serializer
> which also writes out the headers.
>
>
> Thanks,
> Hari
>
> On Tuesday, October 8, 2013 at 7:35 PM, Martinus m wrote:
>
> Hi David,
>
> I'm using Syslog UDP source and on the syslog messages it have it's
> timestamp for each message. I also use HDFS sink, but when I saw the result
> message on HDFS folder, it doesn't have the timestamp (date) info.
>
> Thanks,
>
> Martinus
>
>
> On Tue, Oct 8, 2013 at 9:01 PM, David Sinclair <
> [EMAIL PROTECTED]> wrote:
>
> Martinus,
>
> Can you give a little more details? It sounds like you want to use the
> Spooling Directory Source,
> http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source,
> but if you can be clearer about your requirements, I may be able to help
> you better.
>
> dave
>
>
> On Tue, Oct 8, 2013 at 6:45 AM, Martinus m <[EMAIL PROTECTED]> wrote:
>
> Hi,
>
> What configurations should I put on flume.conf to get the whole logs
> message to be put into the sink?
>
> Thanks.
>
> Martinus
>
>
>
>
>
>
>