Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> issue about how to assiging map output to reducer?


Copy link to this message
-
issue about how to assiging map output to reducer?
hi,maillist:
            i look the containers log from " hadoop fs -cat
/var/log/hadoop-yarn/apps/root/logs/application_1388730279827_2770/CHBM221_50853"

and log say it get 25 map output , and assiging 7 to fetcher 5, assiging 7
to fetcher 4 and assiging 11 to fetcher 3,my question is why not
" assiging 8 to fetcher 5, assiging 8 to fetcher 4 and assiging 9 to
fetcher 3 " ?

2014-01-08 11:28:00,346 INFO [EventFetcher for fetching Map Completion
Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher:
attempt_1388730279827_2770_r_000000_0: Got 25 new map-outputs
2014-01-08 11:28:00,348 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging
CHBM223:8080 with 7 to fetcher#5
2014-01-08 11:28:00,349 INFO [fetcher#5]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 7 of 7
to CHBM223:8080 to fetcher#5
2014-01-08 11:28:00,349 INFO [fetcher#4]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging
CHBM222:8080 with 7 to fetcher#4
2014-01-08 11:28:00,349 INFO [fetcher#4]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 7 of 7
to CHBM222:8080 to fetcher#4
2014-01-08 11:28:00,352 INFO [fetcher#3]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging
CHBM221:8080 with 11 to fetcher#3
2014-01-08 11:28:00,352 INFO [fetcher#3]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 11 of 11
to CHBM221:8080 to fetcher#3
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB