Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Technical question on Capacity Scheduler.


Copy link to this message
-
Re: Technical question on Capacity Scheduler.
Hi All

Can someone please reply to my queries?

On Sun, Mar 3, 2013 at 5:47 PM, Jagmohan Chauhan <[EMAIL PROTECTED]
> wrote:

> Thanks Harsh.
>
> I have a few more questions.
>
> Q1: I found it in my experiments using CS that for any user , its next job
> does not start until its current one is finished. Is it true and are there
> any exceptions and if true then why is it so?  I I did not find any such
> condition in the implementation of CS.
>
> Q2: The concept of reserved slots  is true only if speculative execution
> is on. Am i correct ? If yes,then the code dealing with reserved slots wont
> be executed if speculative execution is off?
>
> PS: I am working on MRv1.
>
>
> On Sun, Mar 3, 2013 at 2:41 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> On Sun, Mar 3, 2013 at 1:41 PM, Jagmohan Chauhan <
>> [EMAIL PROTECTED]
>> > wrote:
>>
>> >  Hi
>> >
>> > I am going through the Capacity Scheduler implementation. There is one
>> > thing i did not understand clearly.
>> >
>>
>> Are you reading the YARN CapacityScheduler or the older, MRv1 one? I'd
>> suggest reading the newer one for any implementation or research goals,
>> for
>> it to be more current and future-applicable.
>>
>>
>> > 1. Does the o ff-switch task refers to a task in which data has to be
>> > fetched over the network. It means its not node-local ?
>> >
>>
>> Off-switch would imply off-rack, i.e. not node local, nor rack-local.
>>
>>
>> > 2. Does off-switch task  includes only the tasks for which map input
>> has to
>> > be fetched from a node on a different rack across the switch or it also
>> > includes task where data has to be fetched from another node on same
>> rack
>> > on same switch?
>> >
>>
>> A task's input split is generally supposed to define all locations of
>> available inputs. If the CS is unable to schedule to any of those
>> locations, nor their racks, then it schedules an off-rack (see above) task
>> which has to pull the input from a different rack.
>>
>>
>> >
>> > --
>> > Thanks and Regards
>> > Jagmohan Chauhan
>> > MSc student,CS
>> > Univ. of Saskatchewan
>> > IEEE Graduate Student Member
>> >
>> > http://homepage.usask.ca/~jac735/
>> >
>>
>> Feel free to post any further impl. related questions! :)
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Thanks and Regards
> Jagmohan Chauhan
> MSc student,CS
> Univ. of Saskatchewan
> IEEE Graduate Student Member
>
> http://homepage.usask.ca/~jac735/
>

--
Thanks and Regards
Jagmohan Chauhan
MSc student,CS
Univ. of Saskatchewan
IEEE Graduate Student Member

http://homepage.usask.ca/~jac735/