Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - HostName intercepter not working -- Anything i am missing


Copy link to this message
-
Re: HostName intercepter not working -- Anything i am missing
Hari Shreedharan 2012-08-29, 09:11
Hi,

The interceptor is probably working as expected. The reason for this is that the serializer that the FILE_ROLL sink uses by default is the BodyText serializer which simply serializes the body. The hostname is in the header though, so you need to write a serializer that can write the header out in the format you want. See : http://flume.apache.org/FlumeUserGuide.html#file-roll-sink. You can write your own serializer and then supply it through the configuration.  
Thanks
Hari
--
Hari Shreedharan
On Wednesday, August 29, 2012 at 1:57 AM, Ashutosh Panigrahi wrote:

> Hi All,
>
> I am a newbie to flume. I have started using it for log aggregation in my company.
>
> I have been desperately trying to make the hostname interceptor work for me , but i am not getting the hostName in the final aggregated log file.
>
> The conf file at my AGGREGATOR machine is
>
> # Describe/configure source1
> agent1.sources.source1.type = avro
> agent1.sources.source1.bind = A.B.C.D
> agent1.sources.source1.port = 44444
>
> agent1.sources.source1.interceptors = host_int inter1
> agent1.sources.source1.interceptors.host_int.type = host
> agent1.sources.source1.interceptors.host_int.hostHeader = hostname
>
> agent1.sources.source1.interceptors.inter1.type = timestamp
>
>
> # Describe sink1
> agent1.sinks.sink1.type = FILE_ROLL
> agent1.sinks.sink1.sink.directory=/home/ashutosh/flumeLogs
> agent1.sinks.sink1.sink.rollInterval=0
>
>
>
> The configuration file at my agents are .
>
> agent1.sources.source1.type = exec
> agent1.sources.source1.command = tail -F /mnt1/logs/LOGGER.log
>
>
> My expectation was that in the Aggregator machine, whatever logs i get, will have the hostname in the beginning. But i am getting the actual string which was generated by agent.
>
> Please help.
>
>
> Regards
> Ashutosh