Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> hadoop 2.0 dependency upgraded to 2.0.3-alpha


Copy link to this message
-
Re: hadoop 2.0 dependency upgraded to 2.0.3-alpha
Thanks for trying to fix it Ted.  I think we need to carry our difficulty
back up to our parent project brethren (Sounds like you have done this
off-list; lets do it in jira/mail too).  Your digging in to nail down the
actual issue will  help making our case.  Maybe we can get/help the
upstream crew make another release (I don't see a note on a new 2.0 release
up on hadoop -- have I missed it)?

Thanks,
St.Ack
On Fri, Feb 22, 2013 at 11:49 AM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Trying out #2, I got:
>
>
> testExcludeAllFromMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
>  Time elapsed: 35.746 sec  <<< FAILURE!
> java.lang.AssertionError
>         at org.junit.Assert.fail(Assert.java:86)
>         at org.junit.Assert.assertTrue(Assert.java:41)
>         at org.junit.Assert.assertTrue(Assert.java:52)
>         at
>
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
>         at
>
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeAllFromMinorCompaction(TestHFileOutputFormat.java:713)
>
> Arun Murthy got to know this defect. Another 2.0 release is coming soon
> (next week, hopefully).
>
> I plan to wait for the new release.
>
> Thanks
>
> On Fri, Feb 22, 2013 at 11:39 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > The root cause was the removal of capacity-scheduler.xml<
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml>
> from
> > 2.0.3-alpha maven artifacts.
> > This config file is needed to start CapacityScheduler which is the
> default.
> >
> > After talking to Sid, I got two suggestions:
> > 1. embed minimal capacity-scheduler.xml<
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml>
> in
> > our src/test/resources
> > 2. use FiFo scheduler instead which doesn't require presence of config
> file
> >
> > I think #2, though temporary fix, is easy to change once a future 2.0
> > release addresses this issue.
> >
> > Once consensus is reached, I will attach another patch to HBASE-7904
> >
> > Cheers
> >
> >
> > On Fri, Feb 22, 2013 at 10:31 AM, Andrew Purtell <[EMAIL PROTECTED]
> >wrote:
> >
> >> For minicluster unit tests? In my opinion, no.
> >>
> >>
> >> On Fri, Feb 22, 2013 at 10:10 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>
> >> > I can try to find the proper usage.
> >> >
> >> > The question is: do we have to maintain compatibility with
> 2.0.2-alpha ?
> >> >
> >> > Thanks
> >> >
> >> > On Fri, Feb 22, 2013 at 10:07 AM, Andrew Purtell <[EMAIL PROTECTED]
> >> > >wrote:
> >> >
> >> > > Sigh. Every release seems to break our uses of MiniMRCluster all
> over
> >> > > again.
> >> > >
> >> > >
> >> > > On Fri, Feb 22, 2013 at 10:04 AM, Ted Yu <[EMAIL PROTECTED]>
> wrote:
> >> > >
> >> > > > I temporarily reverted this change due to:
> >> > > >
> >> > > > org.apache.hadoop.yarn.YarnException: Failed to Start
> >> > > > org.apache.hadoop.mapred.MiniMRCluster
> >> > > >         at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
> >> > > >         at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:75)
> >> > > >         at
> >> > > >
> >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:180)
> >> > > >         at
> >> > > >
> >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:170)
> >> > > >         at
> >> > > >
> >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:162)
> >> > > >         at
> >> > > >
> >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:123)
> >> > > >         at
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1574)