Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: why FairScheduler prefer to schedule MR jobs into the same node?


Copy link to this message
-
Re: why FairScheduler prefer to schedule MR jobs into the same node?
It is possible that you have assignMultiple set to true in your fair
scheduler configuration - that leads to assigning as many tasks on a single
node heartbeat as the node can accommodate. Setting it to false, would
assign a single task on each heartbeat and can help in spreading out the
tasks.
On Fri, Aug 9, 2013 at 4:44 AM, devdoer bird <[EMAIL PROTECTED]> wrote:

> The hadoop version is 1.0.3
>
>
> 2013/8/9 Sandy Ryza <[EMAIL PROTECTED]>
>
>> Hi devdoer,
>>
>> What version are you using?
>>
>> -Sandy
>>
>>
>> On Thu, Aug 8, 2013 at 4:25 AM, devdoer bird <[EMAIL PROTECTED]> wrote:
>>
>>> HI:
>>>
>>> I configure the FairScheduler with default settings and my job has 19
>>>  reduce tasks. I found that all the reduce tasks are schedule to run in one
>>> node.
>>>
>>> While with default FIFO schedule, the 19 reduce tasks are scheduled into
>>> diffrent nodes.
>>>
>>> How can I configure FairSchedule to load more balance?
>>>
>>>
>>> --
>>> devdoer
>>> [EMAIL PROTECTED]
>>>
>>
>>
>
>
> --
> devdoer
> [EMAIL PROTECTED]
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB