Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Technical question on Capacity Scheduler.


Copy link to this message
-
Re: Technical question on Capacity Scheduler.
There are 3 types of locality.

- data local (node local) - your task gets scheduled on the node where the
data is
- rack local - your task is scheduled on the same rack as the data
(usually 20-40 nodes in a rack), but its not on the exact node with the
data.
- off-switch (off-rack) - your task is scheduled on a different rack then
your data is located.

So off-switch it on a different rack. Generally its less expensive to
fetch the data when its in the same rack vs off rack.
Tom

On 3/3/13 2:13 AM, "Jagmohan Chauhan" <[EMAIL PROTECTED]> wrote:

> Hi
>
>I am going through the Capacity Scheduler implementation. There is one
>thing i did not understand clearly.
>1. Does the o ff-switch task refers to a task in which data has to be
>fetched over the network. It means its not node-local ?
>2. Does off-switch task  includes only the tasks for which map input has
>to
>be fetched from a node on a different rack across the switch or it also
>includes task where data has to be fetched from another node on same rack
>on same switch?
>
>--
>Thanks and Regards
>Jagmohan Chauhan
>MSc student,CS
>Univ. of Saskatchewan
>IEEE Graduate Student Member
>
>http://homepage.usask.ca/~jac735/