Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> definition of slots in Hadoop scheduling


Copy link to this message
-
Re: definition of slots in Hadoop scheduling

(Removing common-dev, because this isn't really a dev question)

On Feb 25, 2011, at 5:52 AM, bikash sharma wrote:

> Hi,
> How is task slot in Hadoop defined with respect to scheduling a map/reduce
> task on such slots available on TaskTrackers?
On a TaskTracker, one sets how many maps and reduces one wants to run on that node.  The JobTracker is informed of this value.  When a job is getting scheduled, it compares the various tasks's input to see if a DataNode is providing a matching block.  If a block exists or is nearby, the task is scheduled on that node.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB