Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - How does Hive determine the number of mapred tasks?


Copy link to this message
-
How does Hive determine the number of mapred tasks?
Saurabh Nanda 2010-02-19, 20:52
Hi,

Is there any page/document that describes the methods/techniques used by
Hive to arrive at the optimum number of map tasks & optimum number of reduce
tasks?

I'm running a 3-node Amazon EMR cluster, and Hive has determined that 34 map
& 2 reduce tasks are optimum. Out of the 34 map tasks only 4 are actively
running at any given instant. Any explanations why this exact number?

Saurabh.
--
http://nandz.blogspot.com
http://foodieforlife.blogspot.com