Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - hadoop 2.0 dependency upgraded to 2.0.3-alpha


+
Ted Yu 2013-02-22, 17:35
+
Andrew Purtell 2013-02-22, 17:55
+
Ted Yu 2013-02-22, 18:04
+
Andrew Purtell 2013-02-22, 18:07
+
Ted Yu 2013-02-22, 18:10
+
Andrew Purtell 2013-02-22, 18:31
Copy link to this message
-
Re: hadoop 2.0 dependency upgraded to 2.0.3-alpha
Ted Yu 2013-02-22, 19:39
The root cause was the removal of
capacity-scheduler.xml<http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml>
from
2.0.3-alpha maven artifacts.
This config file is needed to start CapacityScheduler which is the default.

After talking to Sid, I got two suggestions:
1. embed minimal
capacity-scheduler.xml<http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml>
in
our src/test/resources
2. use FiFo scheduler instead which doesn't require presence of config file

I think #2, though temporary fix, is easy to change once a future 2.0
release addresses this issue.

Once consensus is reached, I will attach another patch to HBASE-7904

Cheers

On Fri, Feb 22, 2013 at 10:31 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> For minicluster unit tests? In my opinion, no.
>
>
> On Fri, Feb 22, 2013 at 10:10 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > I can try to find the proper usage.
> >
> > The question is: do we have to maintain compatibility with 2.0.2-alpha ?
> >
> > Thanks
> >
> > On Fri, Feb 22, 2013 at 10:07 AM, Andrew Purtell <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Sigh. Every release seems to break our uses of MiniMRCluster all over
> > > again.
> > >
> > >
> > > On Fri, Feb 22, 2013 at 10:04 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >
> > > > I temporarily reverted this change due to:
> > > >
> > > > org.apache.hadoop.yarn.YarnException: Failed to Start
> > > > org.apache.hadoop.mapred.MiniMRCluster
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:75)
> > > >         at
> > > > org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:180)
> > > >         at
> > > > org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:170)
> > > >         at
> > > > org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:162)
> > > >         at
> > > > org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:123)
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1574)
> > > > ...
> > > > Caused by: org.apache.hadoop.yarn.YarnException:
> > > > java.lang.IllegalStateException: Queue configuration missing child
> > queue
> > > > names for root
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:181)
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
> > > >         ... 38 more
> > > > Caused by: java.lang.IllegalStateException: Queue configuration
> missing
> > > > child queue names for root
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.parseQueue(CapacityScheduler.java:328)
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:255)
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:220)
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.init(ResourceManager.java:226)
> > > >         at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:161)
> > > >
> > > > Cheers
> > > >
> > > > On Fri, Feb 22, 2013 at 9:55 AM, Andrew Purtell <[EMAIL PROTECTED]
> >
> > > > wrote:
+
Ted Yu 2013-02-22, 19:49
+
Stack 2013-02-22, 21:02
+
Ted Yu 2013-02-22, 21:07
+
Andrew Purtell 2013-02-22, 21:12
+
Ted Yu 2013-02-26, 20:03
+
Ted Yu 2013-03-04, 23:56
+
Ted Yu 2013-03-05, 01:04
+
Andrew Purtell 2013-03-05, 00:10
+
Enis Söztutar 2013-02-22, 19:38
+
ramkrishna vasudevan 2013-02-22, 17:56
+
Jean-Marc Spaggiari 2013-02-22, 17:53