See the code for the serializers here:
You can use one of these as an example to write your own.
The configuration documentation for HDFS Sink is here: http://flume.apache.org/FlumeUserGuide.html#hdfs-sink
You can specify a custom serializer by specifying its builder class in the config param "serializer".
On Tuesday, October 16, 2012 at 3:25 PM, Filip Slunecko wrote:
> @Roshan: thanks for suggestion, I will look into it.
> @Hari: I tried to google it up and there is not much about it. I will
> look into it tomorrow and let you know (It' s too late for serious
> work in my time realm :) )
> Thanks both of you for your quick response and help!
> On Wed, Oct 17, 2012 at 12:00 AM, Hari Shreedharan
> <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > Hi Filip,
> > The reason for this is that the Text serializer will only serialized the
> > body of the event, and the syslog sources write the body of the syslog event
> > into the body of the flume event. The hostname/timestamp/severity etc are
> > added into the Flume Event headers. You could simply write a serializer
> > which writes out this information in the same format as you expect and you
> > will be able to see the headers in the files. You could use the Avro
> > serializer to serialize it into avro too, which will make sure the headers
> > are also written out.
> > Hope this helps.
> > Hari
> > --
> > Hari Shreedharan
> > On Tuesday, October 16, 2012 at 2:27 PM, Filip Slunecko wrote:
> > Hi,
> > I am trying to use syslog source and sink it to hdfs or fileroller.
> > Everything is working, but "saved" logs are without timestamp and
> > hostname information.
> > Is it possible to force flume-ng to dump those information from syslog
> > header togather with body lines?
> > I am using flume-ng-agent-1.2.0+24.4-1.noarch from Cloudera repository.
> > Thanks,
> > Filip