Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Hadoop Job Profiling Tool


Copy link to this message
-
Re: Hadoop Job Profiling Tool
Nitin Pawar 2012-07-10, 04:42
job history has most of the details you are looking for at each task levels

On Tue, Jul 10, 2012 at 4:07 AM, Mike S <[EMAIL PROTECTED]> wrote:

> Is there any Hadoop specific tool that can profile a job time? For
> instance, if a job took x amount of time, the tool can profile how
> this time was consumed at different stage likes time spent to read, to
>  map, to reduce reduce , shuttle move that and the different steps of
> a Hadoop M/R job?
>
> If not, how do you recommend finding the map/reduce job bottle neck?
>

--
Nitin Pawar