Hi, all

         I want to do some study on the log of hadoop. I intend to konw how
hadoop record the IO in the logs.

Does anyone do related work and give some suggestions?

Thanks,

May

What are you trying to log/analyse?

I self-assigned to me "do better logging for machine analysis", a long time
ago, but never sat down to do any of it -yet-

https://issues.apache.org/jira/browse/HADOOP-7466
On 23 August 2012 20:44, Li Shengmei <[EMAIL PROTECTED]> wrote:

> Hi, all
>
>          I want to do some study on the log of hadoop. I intend to konw how
> hadoop record the IO in the logs.
>
> Does anyone do related work and give some suggestions?
>
> Thanks,
>
> May
>
>

Hi, Steven,
Thank you very much. I just start to study the log analysis of HADOOP. I will review your work carefully.
Btw, which part of hadoop source codes should be read carefully if I want to get know the log record?

Thanks
May
---------------------------------------------

What are you trying to log/analyse?

I self-assigned to me "do better logging for machine analysis", a long time ago, but never sat down to do any of it -yet-

https://issues.apache.org/jira/browse/HADOOP-7466
On 23 August 2012 20:44, Li Shengmei <[EMAIL PROTECTED]> wrote:

> Hi, all
>
>          I want to do some study on the log of hadoop. I intend to
> konw how hadoop record the IO in the logs.
>
> Does anyone do related work and give some suggestions?
>
> Thanks,
>
> May
>
>

On 29 August 2012 10:36, Li Shengmei <[EMAIL PROTECTED]> wrote:

> Hi, Steven,
>         Thank you very much. I just start to study the log analysis of
> HADOOP. I will review your work carefully.
> Btw, which part of hadoop source codes should be read carefully if I want
> to get know the log record?
>
There's lots of log.info() statements throughout the code -explicit logging
and not any log-injection stuff. If you know what you want to log,
   1. Look at the existing log output at info, and from the name of the log
   that appears in the text (org.apache.hadoop.something.somethingelse)
   2. rummage through the source code in that class to find the bit of
   interest.

The normal log4j settings for Hadoop strip off the full log heirarchy; if
you set the log to a better conversion pattern like
log4j.appender.STDOUT.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} [%t]
%p %c %x: %m%n

then you get more