Hi,

 is there a facility, or a log server, to collect all log generated from
each mapper/reducer of a MR job, in a centralized place, or unified log
file on HDFS?

now i look at logs in a naive way: go to jobtracker to check each task....

 thanks

-xw

Have a look on flume..
On Fri, Jan 11, 2013 at 11:58 PM, Xiaowei Li <[EMAIL PROTECTED]> wrote:

> ct all log generated from
∞
Shashwat Shriparv

To add to that, log aggregation is a feature available with Hadoop 2.0
(where mapreduce is re-written to YARN). The functionality is available via
the History Server:
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HistoryServerRest.html

Thanks
hemanth
On Sat, Jan 12, 2013 at 12:08 AM, shashwat shriparv <
[EMAIL PROTECTED]> wrote:

> Have a look on flume..
>
>
> On Fri, Jan 11, 2013 at 11:58 PM, Xiaowei Li <[EMAIL PROTECTED]> wrote:
>
>> ct all log generated from
>
>
>
>
>
>
> ∞
> Shashwat Shriparv
>
>