-Re: Tracking parts of a job taking the most time
John Meek 2013-06-05, 10:29
hi Ruslan ,
Not sure how to do this? Can you be specific?? Whats DAG? Thanks.
From: Ruslan Al-Fakikh <[EMAIL PROTECTED]>
To: user <[EMAIL PROTECTED]>
Sent: Wed, Jun 5, 2013 4:04 am
Subject: Re: Tracking parts of a job taking the most time
You can look at the Pig script stats after the script is finished. There is
a DAG of MR jobs there. You can look at the individual MR jobs' stats to
see how much time each MR job takes
On Wed, Jun 5, 2013 at 10:15 AM, Johnny Zhang <[EMAIL PROTECTED]> wrote:
> How about disable multi-query execution and use UDF CurrentTime to print
> time between each script block?
> On Tue, Jun 4, 2013 at 7:11 PM, John Meek <[EMAIL PROTECTED]> wrote:
> > All,
> > I have a 400 line pig script which perfoems the calculations I need it to
> > perform, however I need to figure out the amount of time that specific
> > parts of the script take.
> > For example, initial load from a Hbase table - id like to know how much
> > time the load takes before moving onto the next step.
> > Whats the easiest way to break this down?
> > thanks,
> > JM