Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Fair Scheduler is not Fair why?


Copy link to this message
-
Re: Fair Scheduler is not Fair why?
have you enabled task preemption?

Best,

--
Nan Zhu
School of Computer Science,
McGill University

On Wednesday, 16 January, 2013 at 10:45 AM, Justin Workman wrote:

> Looks like weight for both pools is equal and all map slots are used. Therefore I don't believe anyone has priority for the next slots. Try setting research weight to 2. This should allow research to take slots as tech released them.
>
> Sent from my iPhone
>
> On Jan 16, 2013, at 8:26 AM, Dhanasekaran Anbalagan <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
>
> > HI Guys
> >
> > We configured fair scheduler with cdh4, Fair scheduler not work properly.
> > Map Task Capacity = 1380
> > Reduce Task Capacity = 720
> >
> > We create two users tech and research, we configured equal weight 1 But, I stared job in research user mapper will not allocated why?
> > please guide me guys.
> >
> > <?xml version="1.0"?>
> > <allocations>
> > <pool name="tech">
> >   <minMaps>5</minMaps>
> >   <minReduces>5</minReduces>
> >   <maxRunningJobs>30</maxRunningJobs>
> >   <weight>1.0</weight>
> > </pool>
> > <pool name="research">
> >   <minMaps>5</minMaps>
> >   <minReduces>5</minReduces>
> >   <maxRunningJobs>30</maxRunningJobs>
> >   <weight>1.0</weight>
> > </pool>
> > </allocations>
> >
> > Note: we have tested with Hadoop Stream job.
> >
> > Fair Scheduler Administration
> > Pools
> > Pool
> > Running Jobs
> > Map Tasks
> > Reduce Tasks
> > Scheduling Mode
> >
> > Min Share
> > Max Share
> > Running
> > Fair Share
> > Min Share
> > Max Share
> > Running
> > Fair Share
> >
> > research
> > 1
> > 5
> > -
> > 90
> > 690.0
> > 5
> > -
> > 0
> > 0.0
> > FAIR
> >
> > tech
> > 3
> > 5
> > -
> > 1266
> > 690.0
> > 5
> > -
> > 24
> > 24.0
> > FAIR
> >
> > default
> > 0
> > 0
> > -
> > 0
> > 0.0
> > 0
> > -
> > 0
> > 0.0
> > FAIR
> >
> >
> >
> > Running Jobs
> > Submitted
> > JobID
> > User
> > Name
> > Pool
> > Priority
> > Map Tasks
> > Reduce Tasks
> >
> > Finished
> > Running
> > Fair Share
> > Weight
> > Finished
> > Running
> > Fair Share
> > Weight
> >
> > Jan 16, 08:51
> > job_201301071639_2118 (http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2118)
> > tech
> > streamjob5335328828469969152.jar
> >
> >
> > 30466 / 53724
> > 583
> > 313.5
> > 1.0
> > 0 / 24
> > 0
> > 0.0
> > 1.0
> >
> > Jan 16, 09:56
> > job_201301071639_2147 (http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2147)
> > research
> > streamjob8832181817213433660.jar
> >
> >
> > 4175 / 9581
> > 90
> > 690.0
> > 1.0
> > 0 / 24
> > 0
> > 0.0
> > 1.0
> >
> > Jan 16, 10:01
> > job_201301071639_2148 (http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2148)
> > tech
> > streamjob8773848575543653055.jar
> >
> >
> > 1842 / 15484
> > 620
> > 313.5
> > 1.0
> > 0 / 24
> > 0
> > 0.0
> > 1.0
> >
> > Jan 16, 10:08
> > job_201301071639_2155 (http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2155)
> > tech
> > counterfactualsim-prod.eagle-EagleDepthSignalDisabled-prod.eagle
> >
> >
> > 387 / 450
> > 63
> > 63.0
> > 1.0
> > 0 / 24
> > 24
> > 24.0
> > 1.0
> >
> >
> >
> >
> >
> >
> > --
> >  
> >  
> >  

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB