Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Fair Scheduler is not Fair why?


Copy link to this message
-
Re: Fair Scheduler is not Fair why?
HI Nan,

We have not enabled Fair Scheduler Preemption.

-Dhanasekaran.

Did I learn something today? If not, I wasted it.
On Wed, Jan 16, 2013 at 11:21 AM, Nan Zhu <[EMAIL PROTECTED]> wrote:

>  have you enabled task preemption?
>
> Best,
>
> --
> Nan Zhu
> School of Computer Science,
> McGill University
>
>
> On Wednesday, 16 January, 2013 at 10:45 AM, Justin Workman wrote:
>
> Looks like weight for both pools is equal and all map slots are used.
> Therefore I don't believe anyone has priority for the next slots. Try
> setting research weight to 2. This should allow research to take slots as
> tech released them.
>
> Sent from my iPhone
>
> On Jan 16, 2013, at 8:26 AM, Dhanasekaran Anbalagan <[EMAIL PROTECTED]>
> wrote:
>
>  HI Guys
>
> We configured fair scheduler with cdh4, Fair scheduler not work properly.
> Map Task Capacity = 1380
> Reduce Task Capacity = 720
>
> We create two users tech and research, we configured equal weight 1 But, I
> stared job in research user mapper will not allocated why?
> please guide me guys.
>
> <?xml version="1.0"?>
> <allocations>
> <pool name="tech">
>   <minMaps>5</minMaps>
>   <minReduces>5</minReduces>
>   <maxRunningJobs>30</maxRunningJobs>
>   <weight>1.0</weight>
> </pool>
> <pool name="research">
>   <minMaps>5</minMaps>
>   <minReduces>5</minReduces>
>   <maxRunningJobs>30</maxRunningJobs>
>   <weight>1.0</weight>
> </pool>
> </allocations>
>
> Note: we have tested with Hadoop Stream job.
>
> Fair Scheduler Administration Pools PoolRunning JobsMap TasksReduce TasksScheduling
> Mode Min ShareMax ShareRunningFair ShareMin ShareMax ShareRunningFair
> Share research15-90690.05-00.0FAIR tech35-1266690.05-2424.0FAIR default00-
> 00.00-00.0FAIR Running Jobs SubmittedJobIDUserNamePoolPriorityMap TasksReduce
> Tasks FinishedRunningFair ShareWeightFinishedRunningFair ShareWeight Jan
> 16, 08:51 job_201301071639_2118<http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2118>
> tech streamjob5335328828469969152.jar   30466 / 53724583313.5 1.0 0 / 240
> 0.0 1.0 Jan 16, 09:56 job_201301071639_2147<http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2147>
> research streamjob8832181817213433660.jar   4175 / 958190690.0 1.0 0 / 240
> 0.0 1.0 Jan 16, 10:01 job_201301071639_2148<http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2148>
> tech streamjob8773848575543653055.jar   1842 / 15484620313.5 1.0 0 / 240
> 0.0 1.0 Jan 16, 10:08 job_201301071639_2155<http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2155>
> tech counterfactualsim-prod.eagle-EagleDepthSignalDisabled-prod.eagle   387
> / 4506363.0 1.0 0 / 242424.0 1.0
>
> --
>
>
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB