Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Fair Scheduler is not Fair why?


Copy link to this message
-
Re: Fair Scheduler is not Fair why?
Looks like weight for both pools is equal and all map slots are used.
Therefore I don't believe anyone has priority for the next slots. Try
setting research weight to 2. This should allow research to take slots as
tech released them.

Sent from my iPhone

On Jan 16, 2013, at 8:26 AM, Dhanasekaran Anbalagan <[EMAIL PROTECTED]>
wrote:

HI Guys

We configured fair scheduler with cdh4, Fair scheduler not work properly.
Map Task Capacity = 1380
Reduce Task Capacity = 720

We create two users tech and research, we configured equal weight 1 But, I
stared job in research user mapper will not allocated why?
please guide me guys.

<?xml version="1.0"?>
<allocations>
<pool name="tech">
  <minMaps>5</minMaps>
  <minReduces>5</minReduces>
  <maxRunningJobs>30</maxRunningJobs>
  <weight>1.0</weight>
</pool>
<pool name="research">
  <minMaps>5</minMaps>
  <minReduces>5</minReduces>
  <maxRunningJobs>30</maxRunningJobs>
  <weight>1.0</weight>
</pool>
</allocations>

Note: we have tested with Hadoop Stream job.

Fair Scheduler Administration Pools PoolRunning JobsMap TasksReduce
TasksScheduling
Mode Min ShareMax ShareRunningFair ShareMin ShareMax ShareRunningFair Share
research15-90690.05-00.0FAIR tech35-1266690.05-2424.0FAIR default00-00.00-0
0.0FAIR Running Jobs SubmittedJobIDUserNamePoolPriorityMap TasksReduce Tasks
FinishedRunningFair ShareWeightFinishedRunningFair ShareWeight Jan 16, 08:51
job_201301071639_2118<http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2118>
tech streamjob5335328828469969152.jar   30466 / 53724583313.5 1.0 0 / 2400.0
1.0 Jan 16, 09:56
job_201301071639_2147<http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2147>
research streamjob8832181817213433660.jar   4175 / 958190690.0 1.0 0 / 240
0.0 1.0 Jan 16, 10:01
job_201301071639_2148<http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2148>
tech streamjob8773848575543653055.jar   1842 / 15484620313.5 1.0 0 / 2400.0
1.0 Jan 16, 10:08
job_201301071639_2155<http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2155>
tech counterfactualsim-prod.eagle-EagleDepthSignalDisabled-prod.eagle   387
/ 4506363.0 1.0 0 / 242424.0 1.0

--