Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - How to reduce total shuffle time


Copy link to this message
-
How to reduce total shuffle time
Gaurav Dasgupta 2012-08-28, 07:16
Hi,

I have run some large and small jobs and calculated the Total Shuffle Time
for the jobs. I can see that the Total Shuffle Time is almost half the
Total Time which was taken by the full job to complete.

My question, here, is that how can we decrease the Total Shuffle Time? And
doing so, what will be its effect on the Job?

Thanks,
Gaurav Dasgupta