Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> A small portion of map tasks slows down the job


Copy link to this message
-
A small portion of map tasks slows down the job
Hello,

I have a small portion of map tasks whose output is much larger than others (more spills). So the reducer is mainly waiting for these a few map tasks. Is there a good solution for this problem ?

Thank you.

Best,
Huanchen