Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - questions on agents

Copy link to this message
questions on agents
yogi nerella 2013-01-25, 22:12

I have couple of questions on agents.

When a agent goes down?   What happens to the events generated by
application which is using log4jAppender?   Can I specify a channel in the
appender and write them to a local file system, and when the agent is
available, appender can push the data to the agent?

Can the host interceptor intercept the event at the agent and prepend the
source hostname to the event, instead of the hostname of the agent, because
I am collecting log files from multiple applications running on multiple
hostnames, and dont want to modify all applications to write the hostname
to their messages.   Also, dont want to overkill the applications and have
one agent for each application.

I am kind of able to achieve this by running multiple avro sources and
channel each of them to a a different interceptor, but would like if there
is a simple configuration.


Ex: Configuration I am using.

myapp.sources=sock1 sock2 sock3 sock4
myapp.sinks=file1 file2 file3 file4
myapp.channels=mem1 mem2 mem3 mem4


This is kind of limited because it can support only 4 hosts, If there is a
source interceptor we can send all events to a single port and it can
prepend the "hostname" would simplify deployment configuration.