Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> How to Archive the Task Logs (Stdout, Stderr, Syslogs)

Copy link to this message
Re: How to Archive the Task Logs (Stdout, Stderr, Syslogs)

You should be looking at Flume.

Its made for this



Jagat Singh

On Tue, Mar 19, 2013 at 9:03 PM, Christian Schneider <

> Hi,
> I found out that these logs are stored directly at the TaskNodes.
> We need to have them stored over a long time (some months or better a
> year). What is a good way of doing that?
> With my current knowledge I would write a cron job that picks up all the
> files every few minutes.
> But I guess thats not the best approach...
> Best Regards,
> Christian.