Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Technical question on Capacity Scheduler.


Copy link to this message
-
Re: Technical question on Capacity Scheduler.
There are 3 types of locality.

- data local (node local) - your task gets scheduled on the node where the
data is
- rack local - your task is scheduled on the same rack as the data
(usually 20-40 nodes in a rack), but its not on the exact node with the
data.
- off-switch (off-rack) - your task is scheduled on a different rack then
your data is located.

So off-switch it on a different rack. Generally its less expensive to
fetch the data when its in the same rack vs off rack.
Tom

On 3/3/13 2:13 AM, "Jagmohan Chauhan" <[EMAIL PROTECTED]> wrote:

> Hi
>
>I am going through the Capacity Scheduler implementation. There is one
>thing i did not understand clearly.
>1. Does the o ff-switch task refers to a task in which data has to be
>fetched over the network. It means its not node-local ?
>2. Does off-switch task  includes only the tasks for which map input has
>to
>be fetched from a node on a different rack across the switch or it also
>includes task where data has to be fetched from another node on same rack
>on same switch?
>
>--
>Thanks and Regards
>Jagmohan Chauhan
>MSc student,CS
>Univ. of Saskatchewan
>IEEE Graduate Student Member
>
>http://homepage.usask.ca/~jac735/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB