Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Hadoop Job Profiling Tool


Copy link to this message
-
Re: Hadoop Job Profiling Tool
job history has most of the details you are looking for at each task levels

On Tue, Jul 10, 2012 at 4:07 AM, Mike S <[EMAIL PROTECTED]> wrote:

> Is there any Hadoop specific tool that can profile a job time? For
> instance, if a job took x amount of time, the tool can profile how
> this time was consumed at different stage likes time spent to read, to
>  map, to reduce reduce , shuttle move that and the different steps of
> a Hadoop M/R job?
>
> If not, how do you recommend finding the map/reduce job bottle neck?
>

--
Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB