Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> How to Archive the Task Logs (Stdout, Stderr, Syslogs)


+
Christian Schneider 2013-03-19, 10:03
Copy link to this message
-
Re: How to Archive the Task Logs (Stdout, Stderr, Syslogs)
Hello,

You should be looking at Flume.

Its made for this

http://flume.apache.org/

Thanks,

Jagat Singh

On Tue, Mar 19, 2013 at 9:03 PM, Christian Schneider <
[EMAIL PROTECTED]> wrote:

> Hi,
> I found out that these logs are stored directly at the TaskNodes.
>
> We need to have them stored over a long time (some months or better a
> year). What is a good way of doing that?
>
> With my current knowledge I would write a cron job that picks up all the
> files every few minutes.
> But I guess thats not the best approach...
>
> Best Regards,
> Christian.
>
+
Christian Schneider 2013-03-19, 10:42
+
Christian Schneider 2013-03-19, 11:01