Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Map slots and Reduce slots


Copy link to this message
-
RE: Map slots and Reduce slots
Hi Shekar,

   I assume you are trying with Hadoop-1. There are no properties with the names 'mapred.map.max.tasks' and 'mapred.reduce.max.tasks'.

We have these configuration to control the max no of map/reduce tasks run simultaneously.
mapred.tasktracker.map.tasks.maximum - The maximum number of map tasks that will be run simultaneously by a task tracker.
mapred.tasktracker.reduce.tasks.maximum - The maximum number of reduce tasks that will be run simultaneously by a task tracker.

For ex: If we declare mapred.tasktracker.map.tasks.maximum=3  and mapred.tasktracker.reduce.tasks.maximum=4 for a task tracker, means the TT has 3 map slots and 4 reduce slots.

> Let's say on a machine if i have 8GB RAM and dual core machine, how can i determine that what would be the optimal number of map and reducer slots for this machine
It purely depends on which type of tasks you are going to run and load of the task. Normally each task requires one core to execute, no of concurrent tasks can be configured based on this. And memory required for the task depends on how much data it is going to process.
Thanks
Devaraj k

From: Shekhar Sharma [mailto:[EMAIL PROTECTED]]
Sent: 14 July 2013 23:15
To: [EMAIL PROTECTED]
Subject: Map slots and Reduce slots

Does the properties mapred.map.max.tasks=3 and mapred.reduce.max.tasks=4 means that machine has 3 map slots and 4 reduce slots?

Or is there any way i can determine the number of map slots and reduce slots that i can allocate for a machine?

Let's say on a machine if i have 8GB RAM and dual core machine, how can i determine that what would be the optimal number of map and reducer slots for this machine
Regards,
Som Shekhar Sharma
+91-8197243810
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB