Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> questions on agents


Copy link to this message
-
questions on agents
Hi,

I have couple of questions on agents.

When a agent goes down?   What happens to the events generated by
application which is using log4jAppender?   Can I specify a channel in the
appender and write them to a local file system, and when the agent is
available, appender can push the data to the agent?

Can the host interceptor intercept the event at the agent and prepend the
source hostname to the event, instead of the hostname of the agent, because
I am collecting log files from multiple applications running on multiple
hostnames, and dont want to modify all applications to write the hostname
to their messages.   Also, dont want to overkill the applications and have
one agent for each application.

I am kind of able to achieve this by running multiple avro sources and
channel each of them to a a different interceptor, but would like if there
is a simple configuration.

Thanks,
Yogi

Ex: Configuration I am using.

myapp.sources=sock1 sock2 sock3 sock4
myapp.sinks=file1 file2 file3 file4
myapp.channels=mem1 mem2 mem3 mem4

sock1.port=44441
sock2.port=44442
sock3.port=44443
sock4.port=44444

This is kind of limited because it can support only 4 hosts, If there is a
source interceptor we can send all events to a single port and it can
prepend the "hostname" would simplify deployment configuration.