Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> hadoop 2.0 dependency upgraded to 2.0.3-alpha


Copy link to this message
-
Re: hadoop 2.0 dependency upgraded to 2.0.3-alpha
Here is another blocker for running mapreduce on top hadoop 2.0.3-alpha:

https://issues.apache.org/jira/browse/YARN-449

On Tue, Feb 26, 2013 at 12:03 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> https://issues.apache.org/jira/browse/YARN-429 was logged for this issue.
>
> FYI
>
>
> On Fri, Feb 22, 2013 at 1:12 PM, Andrew Purtell <[EMAIL PROTECTED]>wrote:
>
>> I agree, thanks a lot for doing this.
>>
>>
>> On Fri, Feb 22, 2013 at 1:07 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > bq. to nail down the actual issue will  help making our case.
>> >
>> > That is what I am trying to do. Will ping Arun at the meetup this
>> afternoon
>> > about when the next release is coming.
>> >
>> > Cheers
>> >
>> > On Fri, Feb 22, 2013 at 1:02 PM, Stack <[EMAIL PROTECTED]> wrote:
>> >
>> > > Thanks for trying to fix it Ted.  I think we need to carry our
>> difficulty
>> > > back up to our parent project brethren (Sounds like you have done this
>> > > off-list; lets do it in jira/mail too).  Your digging in to nail down
>> the
>> > > actual issue will  help making our case.  Maybe we can get/help the
>> > > upstream crew make another release (I don't see a note on a new 2.0
>> > release
>> > > up on hadoop -- have I missed it)?
>> > >
>> > > Thanks,
>> > > St.Ack
>> > >
>> > >
>> > > On Fri, Feb 22, 2013 at 11:49 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> > >
>> > > > Trying out #2, I got:
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> testExcludeAllFromMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
>> > > >  Time elapsed: 35.746 sec  <<< FAILURE!
>> > > > java.lang.AssertionError
>> > > >         at org.junit.Assert.fail(Assert.java:86)
>> > > >         at org.junit.Assert.assertTrue(Assert.java:41)
>> > > >         at org.junit.Assert.assertTrue(Assert.java:52)
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
>> > > >         at
>> > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeAllFromMinorCompaction(TestHFileOutputFormat.java:713)
>> > > >
>> > > > Arun Murthy got to know this defect. Another 2.0 release is coming
>> soon
>> > > > (next week, hopefully).
>> > > >
>> > > > I plan to wait for the new release.
>> > > >
>> > > > Thanks
>> > > >
>> > > > On Fri, Feb 22, 2013 at 11:39 AM, Ted Yu <[EMAIL PROTECTED]>
>> wrote:
>> > > >
>> > > > > The root cause was the removal of capacity-scheduler.xml<
>> > > >
>> > >
>> >
>> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml
>> > > >
>> > > > from
>> > > > > 2.0.3-alpha maven artifacts.
>> > > > > This config file is needed to start CapacityScheduler which is the
>> > > > default.
>> > > > >
>> > > > > After talking to Sid, I got two suggestions:
>> > > > > 1. embed minimal capacity-scheduler.xml<
>> > > >
>> > >
>> >
>> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml
>> > > >
>> > > > in
>> > > > > our src/test/resources
>> > > > > 2. use FiFo scheduler instead which doesn't require presence of
>> > config
>> > > > file
>> > > > >
>> > > > > I think #2, though temporary fix, is easy to change once a future
>> 2.0
>> > > > > release addresses this issue.
>> > > > >
>> > > > > Once consensus is reached, I will attach another patch to
>> HBASE-7904
>> > > > >
>> > > > > Cheers
>> > > > >
>> > > > >
>> > > > > On Fri, Feb 22, 2013 at 10:31 AM, Andrew Purtell <
>> > [EMAIL PROTECTED]
>> > > > >wrote:
>> > > > >
>> > > > >> For minicluster unit tests? In my opinion, no.
>> > > > >>
>> > > > >>
>> > > > >> On Fri, Feb 22, 2013 at 10:10 AM, Ted Yu <[EMAIL PROTECTED]>
>> > wrote:
>> > > > >>
>> > > > >> > I can try to find the proper usage.
>> > > > >> >
>> > > > >> > The question is: do we have to maintain compatibility with
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB