Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - flume-ng failure recovery


Copy link to this message
-
flume-ng failure recovery
Jagadish Bihani 2012-07-18, 05:53
Hi

We want to deploy flume-ng in the production environment in our
organization.
Here is the following scenario for which I am not able to find the answer:

1. We receive logs using 'tail -f' source.
2. Now  the agent process gets killed.
3. We restart it.
4. How would the restarted agent will know the correct state of the file.
Because in the meantime log file would have been modified and agent has
no way of knowing from where to resume?

Could you please help me in identifying how to tackle this scenario?

P.S. Instead of tail -f any other command can be used which doesnt
modify the log file.

Regards,
Jagadish