Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - hadoop 2.0 dependency upgraded to 2.0.3-alpha


Copy link to this message
-
Re: hadoop 2.0 dependency upgraded to 2.0.3-alpha
Ted Yu 2013-02-22, 19:49
Trying out #2, I got:

testExcludeAllFromMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
 Time elapsed: 35.746 sec  <<< FAILURE!
java.lang.AssertionError
        at org.junit.Assert.fail(Assert.java:86)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at org.junit.Assert.assertTrue(Assert.java:52)
        at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
        at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeAllFromMinorCompaction(TestHFileOutputFormat.java:713)

Arun Murthy got to know this defect. Another 2.0 release is coming soon
(next week, hopefully).

I plan to wait for the new release.

Thanks

On Fri, Feb 22, 2013 at 11:39 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> The root cause was the removal of capacity-scheduler.xml<http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml> from
> 2.0.3-alpha maven artifacts.
> This config file is needed to start CapacityScheduler which is the default.
>
> After talking to Sid, I got two suggestions:
> 1. embed minimal capacity-scheduler.xml<http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml> in
> our src/test/resources
> 2. use FiFo scheduler instead which doesn't require presence of config file
>
> I think #2, though temporary fix, is easy to change once a future 2.0
> release addresses this issue.
>
> Once consensus is reached, I will attach another patch to HBASE-7904
>
> Cheers
>
>
> On Fri, Feb 22, 2013 at 10:31 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:
>
>> For minicluster unit tests? In my opinion, no.
>>
>>
>> On Fri, Feb 22, 2013 at 10:10 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > I can try to find the proper usage.
>> >
>> > The question is: do we have to maintain compatibility with 2.0.2-alpha ?
>> >
>> > Thanks
>> >
>> > On Fri, Feb 22, 2013 at 10:07 AM, Andrew Purtell <[EMAIL PROTECTED]
>> > >wrote:
>> >
>> > > Sigh. Every release seems to break our uses of MiniMRCluster all over
>> > > again.
>> > >
>> > >
>> > > On Fri, Feb 22, 2013 at 10:04 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> > >
>> > > > I temporarily reverted this change due to:
>> > > >
>> > > > org.apache.hadoop.yarn.YarnException: Failed to Start
>> > > > org.apache.hadoop.mapred.MiniMRCluster
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:75)
>> > > >         at
>> > > >
>> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:180)
>> > > >         at
>> > > >
>> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:170)
>> > > >         at
>> > > >
>> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:162)
>> > > >         at
>> > > >
>> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:123)
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1574)
>> > > > ...
>> > > > Caused by: org.apache.hadoop.yarn.YarnException:
>> > > > java.lang.IllegalStateException: Queue configuration missing child
>> > queue
>> > > > names for root
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:181)
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>> > > >         ... 38 more
>> > > > Caused by: java.lang.IllegalStateException: Queue configuration
>> missing
>> > > > child queue names for root
>> >