-
Re: JobTracker scheduling when there is no locality optimization.Harsh J 2012-12-07, 22:25
It is currently 'random' (based on incoming tasktracker heartbeat order and
availability of slots). MR2 and MR1 with CapacityScheduler can utilize
memory resource request parameters in the job config to schedule based
on memory-constraints, which is the highest you could do at the moment
(CPU based scheduling is arriving soon for YARN, though).
On Sat, Dec 8, 2012 at 3:39 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> How does the JobTracker schedule / assign tasks to open TaskTrackers when
> there is no locality constraints (i.e. when some non file input format,
> such as DBInputFormat is used).... Is it random? Or can it optimize for IO?
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
--
Harsh J
availability of slots). MR2 and MR1 with CapacityScheduler can utilize
memory resource request parameters in the job config to schedule based
on memory-constraints, which is the highest you could do at the moment
(CPU based scheduling is arriving soon for YARN, though).
On Sat, Dec 8, 2012 at 3:39 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> How does the JobTracker schedule / assign tasks to open TaskTrackers when
> there is no locality constraints (i.e. when some non file input format,
> such as DBInputFormat is used).... Is it random? Or can it optimize for IO?
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
--
Harsh J
