Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> How does Hive determine the number of mapred tasks?


Copy link to this message
-
How does Hive determine the number of mapred tasks?
Hi,

Is there any page/document that describes the methods/techniques used by
Hive to arrive at the optimum number of map tasks & optimum number of reduce
tasks?

I'm running a 3-node Amazon EMR cluster, and Hive has determined that 34 map
& 2 reduce tasks are optimum. Out of the 34 map tasks only 4 are actively
running at any given instant. Any explanations why this exact number?

Saurabh.
--
http://nandz.blogspot.com
http://foodieforlife.blogspot.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB