Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> How to Archive the Task Logs (Stdout, Stderr, Syslogs)


+
Christian Schneider 2013-03-19, 10:03
+
Jagat Singh 2013-03-19, 10:05
Copy link to this message
-
Re: How to Archive the Task Logs (Stdout, Stderr, Syslogs)
Hi Jagat,
Thank you. That sounds good. I will have a  look at it.

Best Regards,
Christian.
2013/3/19 Jagat Singh <[EMAIL PROTECTED]>

> Hello,
>
> You should be looking at Flume.
>
> Its made for this
>
> http://flume.apache.org/
>
> Thanks,
>
> Jagat Singh
>
>
> On Tue, Mar 19, 2013 at 9:03 PM, Christian Schneider <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>> I found out that these logs are stored directly at the TaskNodes.
>>
>> We need to have them stored over a long time (some months or better a
>> year). What is a good way of doing that?
>>
>> With my current knowledge I would write a cron job that picks up all the
>> files every few minutes.
>> But I guess thats not the best approach...
>>
>> Best Regards,
>> Christian.
>>
>
>
+
Christian Schneider 2013-03-19, 11:01