Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> 33%, 66%, and 100% *reducer* optimization


Copy link to this message
-
Re: 33%, 66%, and 100% *reducer* optimization
Reduce has three phases - shuffle, sort and reduce.

So, 33% would imply the shuffle phase end, and 66% would refer to the end of sort phase.

Thanks,
+Vinod

On Oct 15, 2012, at 2:32 PM, Jay Vyas wrote:

> Hi guys !
>
> We all know that there are major milestones in reducers (33%, 66%)
>
> In particular:
>
> Im wondering: What should we optimize for in case a job hangs at 33%, as
> compared with 66% ?
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB