Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - HostName intercepter not working -- Anything i am missing


Copy link to this message
-
Re: HostName intercepter not working -- Anything i am missing
Mike Percy 2012-08-29, 09:29
Worth trying out the AVRO_EVENT serializer which comes out of the box,
which writes the data in Avro container format (including all headers).

Regards,
Mike

On Wed, Aug 29, 2012 at 2:11 AM, Hari Shreedharan <[EMAIL PROTECTED]
> wrote:

> Hi,
>
> The interceptor is probably working as expected. The reason for this is
> that the serializer that the FILE_ROLL sink uses by default is the BodyText
> serializer which simply serializes the body. The hostname is in the header
> though, so you need to write a serializer that can write the header out in
> the format you want. See :
> http://flume.apache.org/FlumeUserGuide.html#file-roll-sink. You can write
> your own serializer and then supply it through the configuration.
>
>
> Thanks
> Hari
> --
> Hari Shreedharan
>
>
> On Wednesday, August 29, 2012 at 1:57 AM, Ashutosh Panigrahi wrote:
>
> > Hi All,
> >
> > I am a newbie to flume. I have started using it for log aggregation in
> my company.
> >
> > I have been desperately trying to make the hostname interceptor work for
> me , but i am not getting the hostName in the final aggregated log file.
> >
> > The conf file at my AGGREGATOR machine is
> >
> > # Describe/configure source1
> > agent1.sources.source1.type = avro
> > agent1.sources.source1.bind = A.B.C.D
> > agent1.sources.source1.port = 44444
> >
> > agent1.sources.source1.interceptors = host_int inter1
> > agent1.sources.source1.interceptors.host_int.type = host
> > agent1.sources.source1.interceptors.host_int.hostHeader = hostname
> >
> > agent1.sources.source1.interceptors.inter1.type = timestamp
> >
> >
> > # Describe sink1
> > agent1.sinks.sink1.type = FILE_ROLL
> > agent1.sinks.sink1.sink.directory=/home/ashutosh/flumeLogs
> > agent1.sinks.sink1.sink.rollInterval=0
> >
> >
> >
> > The configuration file at my agents are .
> >
> > agent1.sources.source1.type = exec
> > agent1.sources.source1.command = tail -F /mnt1/logs/LOGGER.log
> >
> >
> > My expectation was that in the Aggregator machine, whatever logs i get,
> will have the hostname in the beginning. But i am getting the actual string
> which was generated by agent.
> >
> > Please help.
> >
> >
> > Regards
> > Ashutosh
>
>
>
>