Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Archive Task Logs (Stdout, Stderr, Sysogs) and Job Tracker logs of a Hadoop Cluster for later analysis


Copy link to this message
-
Archive Task Logs (Stdout, Stderr, Sysogs) and Job Tracker logs of a Hadoop Cluster for later analysis
Christian Schneider 2013-04-08, 14:33
Hi,
I need to collect log data from our Cluster.

For this I think I need to copy the Contents of:
* JobTracker: /var/log/hadoop-0.20-mapreduce/history/
* TaskTracker: /var/log/hadoop-0.20-mapreduce/userlogs/

It should also follow symlinks and copy recusrive.

Is flume the right tool to do this?

E.g. with the "Spooling Directory Source"?

Best Regards,
Christian.