Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Map Reduce slot


Copy link to this message
-
Re: Map Reduce slot
Hi Udippan

The number of map/reduce tasks slots on your cluster is the sum of all the
slots from your TaskTracker nodes. Based on your node's resource
availability you can even configure that on a per node basis. The slots are
defined on a node level using the following properties
mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum .

As Kapil mentioned the total number of slots across the cluster can be
obtained from JT web UI.

The other 3 properties can be defined on a job level. However in production
clustes the jvm size is marked final to prevent abuses that may lead to
OOMs.

The jvm size of task jvms are defined by 'mapred.child.java.opts' which
defaults to 200 Mb and the jvm reuse is defined by
'mapred.job.reuse.jvm.num.tasks' which defaults to 1 task per jvm.

HTH.

Regards
Bejoy KS
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB