Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> issue about how to assiging map output to reducer?


Copy link to this message
-
issue about how to assiging map output to reducer?
hi,maillist:
            i look the containers log from " hadoop fs -cat
/var/log/hadoop-yarn/apps/root/logs/application_1388730279827_2770/CHBM221_50853"

and log say it get 25 map output , and assiging 7 to fetcher 5, assiging 7
to fetcher 4 and assiging 11 to fetcher 3,my question is why not
" assiging 8 to fetcher 5, assiging 8 to fetcher 4 and assiging 9 to
fetcher 3 " ?

2014-01-08 11:28:00,346 INFO [EventFetcher for fetching Map Completion
Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher:
attempt_1388730279827_2770_r_000000_0: Got 25 new map-outputs
2014-01-08 11:28:00,348 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging
CHBM223:8080 with 7 to fetcher#5
2014-01-08 11:28:00,349 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 7 of 7
to CHBM223:8080 to fetcher#5
2014-01-08 11:28:00,349 INFO [fetcher#4]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging
CHBM222:8080 with 7 to fetcher#4
2014-01-08 11:28:00,349 INFO [fetcher#4]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 7 of 7
to CHBM222:8080 to fetcher#4
2014-01-08 11:28:00,352 INFO [fetcher#3]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging
CHBM221:8080 with 11 to fetcher#3
2014-01-08 11:28:00,352 INFO [fetcher#3]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 11 of 11
to CHBM221:8080 to fetcher#3