Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> hadoop 2.0 dependency upgraded to 2.0.3-alpha


+
Ted Yu 2013-02-22, 17:35
+
Andrew Purtell 2013-02-22, 17:55
+
Ted Yu 2013-02-22, 18:04
+
Andrew Purtell 2013-02-22, 18:07
+
Ted Yu 2013-02-22, 18:10
+
Andrew Purtell 2013-02-22, 18:31
+
Ted Yu 2013-02-22, 19:39
Copy link to this message
-
Re: hadoop 2.0 dependency upgraded to 2.0.3-alpha
Trying out #2, I got:

testExcludeAllFromMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
 Time elapsed: 35.746 sec  <<< FAILURE!
java.lang.AssertionError
        at org.junit.Assert.fail(Assert.java:86)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at org.junit.Assert.assertTrue(Assert.java:52)
        at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
        at
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeAllFromMinorCompaction(TestHFileOutputFormat.java:713)

Arun Murthy got to know this defect. Another 2.0 release is coming soon
(next week, hopefully).

I plan to wait for the new release.

Thanks

On Fri, Feb 22, 2013 at 11:39 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> The root cause was the removal of capacity-scheduler.xml<http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml> from
> 2.0.3-alpha maven artifacts.
> This config file is needed to start CapacityScheduler which is the default.
>
> After talking to Sid, I got two suggestions:
> 1. embed minimal capacity-scheduler.xml<http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml> in
> our src/test/resources
> 2. use FiFo scheduler instead which doesn't require presence of config file
>
> I think #2, though temporary fix, is easy to change once a future 2.0
> release addresses this issue.
>
> Once consensus is reached, I will attach another patch to HBASE-7904
>
> Cheers
>
>
> On Fri, Feb 22, 2013 at 10:31 AM, Andrew Purtell <[EMAIL PROTECTED]>wrote:
>
>> For minicluster unit tests? In my opinion, no.
>>
>>
>> On Fri, Feb 22, 2013 at 10:10 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > I can try to find the proper usage.
>> >
>> > The question is: do we have to maintain compatibility with 2.0.2-alpha ?
>> >
>> > Thanks
>> >
>> > On Fri, Feb 22, 2013 at 10:07 AM, Andrew Purtell <[EMAIL PROTECTED]
>> > >wrote:
>> >
>> > > Sigh. Every release seems to break our uses of MiniMRCluster all over
>> > > again.
>> > >
>> > >
>> > > On Fri, Feb 22, 2013 at 10:04 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> > >
>> > > > I temporarily reverted this change due to:
>> > > >
>> > > > org.apache.hadoop.yarn.YarnException: Failed to Start
>> > > > org.apache.hadoop.mapred.MiniMRCluster
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:75)
>> > > >         at
>> > > >
>> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:180)
>> > > >         at
>> > > >
>> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:170)
>> > > >         at
>> > > >
>> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:162)
>> > > >         at
>> > > >
>> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:123)
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1574)
>> > > > ...
>> > > > Caused by: org.apache.hadoop.yarn.YarnException:
>> > > > java.lang.IllegalStateException: Queue configuration missing child
>> > queue
>> > > > names for root
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:181)
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
>> > > >         ... 38 more
>> > > > Caused by: java.lang.IllegalStateException: Queue configuration
>> missing
>> > > > child queue names for root
>> >
+
Stack 2013-02-22, 21:02
+
Ted Yu 2013-02-22, 21:07
+
Andrew Purtell 2013-02-22, 21:12
+
Ted Yu 2013-02-26, 20:03
+
Ted Yu 2013-03-04, 23:56
+
Ted Yu 2013-03-05, 01:04
+
Andrew Purtell 2013-03-05, 00:10
+
Enis Söztutar 2013-02-22, 19:38
+
ramkrishna vasudevan 2013-02-22, 17:56
+
Jean-Marc Spaggiari 2013-02-22, 17:53
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB