Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - 33%, 66%, and 100% *reducer* optimization


+
Jay Vyas 2012-10-15, 21:32
Copy link to this message
-
Re: 33%, 66%, and 100% *reducer* optimization
Vinod Kumar Vavilapalli 2012-10-15, 23:33
Reduce has three phases - shuffle, sort and reduce.

So, 33% would imply the shuffle phase end, and 66% would refer to the end of sort phase.

Thanks,
+Vinod

On Oct 15, 2012, at 2:32 PM, Jay Vyas wrote:

> Hi guys !
>
> We all know that there are major milestones in reducers (33%, 66%)
>
> In particular:
>
> Im wondering: What should we optimize for in case a job hangs at 33%, as
> compared with 66% ?
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com