Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> hadoop 2.0 dependency upgraded to 2.0.3-alpha


Copy link to this message
-
Re: hadoop 2.0 dependency upgraded to 2.0.3-alpha
Thanks for trying to fix it Ted.  I think we need to carry our difficulty
back up to our parent project brethren (Sounds like you have done this
off-list; lets do it in jira/mail too).  Your digging in to nail down the
actual issue will  help making our case.  Maybe we can get/help the
upstream crew make another release (I don't see a note on a new 2.0 release
up on hadoop -- have I missed it)?

Thanks,
St.Ack
On Fri, Feb 22, 2013 at 11:49 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Trying out #2, I got:
>
>
> testExcludeAllFromMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
>  Time elapsed: 35.746 sec  <<< FAILURE!
> java.lang.AssertionError
>         at org.junit.Assert.fail(Assert.java:86)
>         at org.junit.Assert.assertTrue(Assert.java:41)
>         at org.junit.Assert.assertTrue(Assert.java:52)
>         at
>
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
>         at
>
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeAllFromMinorCompaction(TestHFileOutputFormat.java:713)
>
> Arun Murthy got to know this defect. Another 2.0 release is coming soon
> (next week, hopefully).
>
> I plan to wait for the new release.
>
> Thanks
>
> On Fri, Feb 22, 2013 at 11:39 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > The root cause was the removal of capacity-scheduler.xml<
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml>
> from
> > 2.0.3-alpha maven artifacts.
> > This config file is needed to start CapacityScheduler which is the
> default.
> >
> > After talking to Sid, I got two suggestions:
> > 1. embed minimal capacity-scheduler.xml<
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml>
> in
> > our src/test/resources
> > 2. use FiFo scheduler instead which doesn't require presence of config
> file
> >
> > I think #2, though temporary fix, is easy to change once a future 2.0
> > release addresses this issue.
> >
> > Once consensus is reached, I will attach another patch to HBASE-7904
> >
> > Cheers
> >
> >
> > On Fri, Feb 22, 2013 at 10:31 AM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
> >
> >> For minicluster unit tests? In my opinion, no.
> >>
> >>
> >> On Fri, Feb 22, 2013 at 10:10 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>
> >> > I can try to find the proper usage.
> >> >
> >> > The question is: do we have to maintain compatibility with
> 2.0.2-alpha ?
> >> >
> >> > Thanks
> >> >
> >> > On Fri, Feb 22, 2013 at 10:07 AM, Andrew Purtell <[EMAIL PROTECTED]
> >> > >wrote:
> >> >
> >> > > Sigh. Every release seems to break our uses of MiniMRCluster all
> over
> >> > > again.
> >> > >
> >> > >
> >> > > On Fri, Feb 22, 2013 at 10:04 AM, Ted Yu <[EMAIL PROTECTED]>
> wrote:
> >> > >
> >> > > > I temporarily reverted this change due to:
> >> > > >
> >> > > > org.apache.hadoop.yarn.YarnException: Failed to Start
> >> > > > org.apache.hadoop.mapred.MiniMRCluster
> >> > > >         at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
> >> > > >         at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:75)
> >> > > >         at
> >> > > >
> >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:180)
> >> > > >         at
> >> > > >
> >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:170)
> >> > > >         at
> >> > > >
> >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:162)
> >> > > >         at
> >> > > >
> >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:123)
> >> > > >         at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1574)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB