Are there any utilities available to help parse jobtracker log files?

Hank Cohen

[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
www.altior.com<http://www.altior.com>

Now, no utilities for job tracker log .  
--  
peter
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On 2012年8月17日Friday at 下午12:29, Hank Cohen wrote:

> Are there any utilities available to help parse jobtracker log files?
>  
> Hank Cohen
>  
> [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])
> www.altior.com (http://www.altior.com)
>  
>  
>  
>  
>  
>  

Take a look at Pig's HadoopJobHistoryLoader
http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/piggybank/storage/HadoopJobHistoryLoader.html

On Thu, Aug 16, 2012 at 9:34 PM, peter <[EMAIL PROTECTED]> wrote:

> Now, no utilities for job tracker log .
>
>
> --
> peter
> Sent with Sparrow <http://www.sparrowmailapp.com/?sig>
>
> On 2012年8月17日Friday at 下午12:29, Hank Cohen wrote:
>
>  Are there any utilities available to help parse jobtracker log files?****
>
> ** **
>
> *Hank Cohen*****
>
> ** **
>
> [EMAIL PROTECTED]****
>
> www.altior.com****
>
> ** **
>
> ** **
>
> ** **
>
>
>

Hey Hank,

What are you looking to extract out of this service log exactly?

On Fri, Aug 17, 2012 at 9:59 AM, Hank Cohen <[EMAIL PROTECTED]> wrote:
> Are there any utilities available to help parse jobtracker log files?
>
>
>
> Hank Cohen
>
>
>
> [EMAIL PROTECTED]
>
> www.altior.com
>
>
>
>
>
>

--
Harsh J

Hi,

I would also look into logstash if you are looking for analyzing logs
across your cluster:

http://logstash.net/docs/1.1.1/

I cannot recommend it from experience, but it seems to be a tool built for
this type of problem.

Best,
Ariel
On Aug 17, 2012 12:52 AM, "Harsh J" <[EMAIL PROTECTED]> wrote:

> Hey Hank,
>
> What are you looking to extract out of this service log exactly?
>
> On Fri, Aug 17, 2012 at 9:59 AM, Hank Cohen <[EMAIL PROTECTED]> wrote:
> > Are there any utilities available to help parse jobtracker log files?
> >
> >
> >
> > Hank Cohen
> >
> >
> >
> > [EMAIL PROTECTED]
> >
> > www.altior.com
> >
> >
> >
> >
> >
> >
>
>
>
> --
> Harsh J
>

org.apache.hadoop.mapred.DefaultJobHistoryParser is a good place to start.

On Fri, Aug 17, 2012 at 12:29 AM, Hank Cohen <[EMAIL PROTECTED]> wrote:
> Are there any utilities available to help parse jobtracker log files?
>
>
>
> Hank Cohen
>
>
>
> [EMAIL PROTECTED]
>
> www.altior.com
>
>
>
>
>
>

if you want to parse job log files, you can take a look at

https://github.com/rajvish/hadoop-summary

Raj 

>________________________________
> From: Stan Rosenberg <[EMAIL PROTECTED]>
>To: [EMAIL PROTECTED]
>Sent: Friday, August 17, 2012 9:59 AM
>Subject: Re: Log file parsing
>
>org.apache.hadoop.mapred.DefaultJobHistoryParser is a good place to start.
>
>On Fri, Aug 17, 2012 at 12:29 AM, Hank Cohen <[EMAIL PROTECTED]> wrote:
>> Are there any utilities available to help parse jobtracker log files?
>>
>>
>>
>> Hank Cohen
>>
>>
>>
>> [EMAIL PROTECTED]
>>
>> www.altior.com
>>
>>
>>
>>
>>
>>
>
>
>

Raj,
Thank you! This is perfect.
I was about to learn Perl so I could write a similar program.  I don't think awk or sed would cut it.
Now I don't have to and I can get on to my analysis right away.

Hank Cohen

From: Raj Vishwanathan [mailto:[EMAIL PROTECTED]]
Sent: Friday, August 17, 2012 3:34 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: Log file parsing

if you want to parse job log files, you can take a look at

https://github.com/rajvish/hadoop-summary

Raj

________________________________
From: Stan Rosenberg <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
Sent: Friday, August 17, 2012 9:59 AM
Subject: Re: Log file parsing

org.apache.hadoop.mapred.DefaultJobHistoryParser is a good place to start.

On Fri, Aug 17, 2012 at 12:29 AM, Hank Cohen <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
> Are there any utilities available to help parse jobtracker log files?
>
>
>
> Hank Cohen
>
>
>
> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
>
> www.altior.com<http://www.altior.com>
>
>
>
>
>
>