Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - How to Archive the Task Logs (Stdout, Stderr, Syslogs)


Copy link to this message
-
How to Archive the Task Logs (Stdout, Stderr, Syslogs)
Christian Schneider 2013-03-19, 10:03
Hi,
I found out that these logs are stored directly at the TaskNodes.

We need to have them stored over a long time (some months or better a
year). What is a good way of doing that?

With my current knowledge I would write a cron job that picks up all the
files every few minutes.
But I guess thats not the best approach...

Best Regards,
Christian.