Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - hadoop 2.0 dependency upgraded to 2.0.3-alpha


+
Ted Yu 2013-02-22, 17:35
+
Andrew Purtell 2013-02-22, 17:55
+
Ted Yu 2013-02-22, 18:04
+
Andrew Purtell 2013-02-22, 18:07
+
Ted Yu 2013-02-22, 18:10
+
Andrew Purtell 2013-02-22, 18:31
+
Ted Yu 2013-02-22, 19:39
+
Ted Yu 2013-02-22, 19:49
+
Stack 2013-02-22, 21:02
+
Ted Yu 2013-02-22, 21:07
+
Andrew Purtell 2013-02-22, 21:12
+
Ted Yu 2013-02-26, 20:03
+
Ted Yu 2013-03-04, 23:56
Copy link to this message
-
Re: hadoop 2.0 dependency upgraded to 2.0.3-alpha
Ted Yu 2013-03-05, 01:04
I seem to find a way to get mapreduce job to run (with fix for
YARN-429<https://issues.apache.org/jira/browse/YARN-429> in
place).

See my attachment in YARN-449<https://issues.apache.org/jira/browse/YARN-449>

FYI

On Mon, Mar 4, 2013 at 3:56 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Here is another blocker for running mapreduce on top hadoop 2.0.3-alpha:
>
> https://issues.apache.org/jira/browse/YARN-449
>
>
> On Tue, Feb 26, 2013 at 12:03 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> https://issues.apache.org/jira/browse/YARN-429 was logged for this issue.
>>
>> FYI
>>
>>
>> On Fri, Feb 22, 2013 at 1:12 PM, Andrew Purtell <[EMAIL PROTECTED]>wrote:
>>
>>> I agree, thanks a lot for doing this.
>>>
>>>
>>> On Fri, Feb 22, 2013 at 1:07 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>>
>>> > bq. to nail down the actual issue will  help making our case.
>>> >
>>> > That is what I am trying to do. Will ping Arun at the meetup this
>>> afternoon
>>> > about when the next release is coming.
>>> >
>>> > Cheers
>>> >
>>> > On Fri, Feb 22, 2013 at 1:02 PM, Stack <[EMAIL PROTECTED]> wrote:
>>> >
>>> > > Thanks for trying to fix it Ted.  I think we need to carry our
>>> difficulty
>>> > > back up to our parent project brethren (Sounds like you have done
>>> this
>>> > > off-list; lets do it in jira/mail too).  Your digging in to nail
>>> down the
>>> > > actual issue will  help making our case.  Maybe we can get/help the
>>> > > upstream crew make another release (I don't see a note on a new 2.0
>>> > release
>>> > > up on hadoop -- have I missed it)?
>>> > >
>>> > > Thanks,
>>> > > St.Ack
>>> > >
>>> > >
>>> > > On Fri, Feb 22, 2013 at 11:49 AM, Ted Yu <[EMAIL PROTECTED]>
>>> wrote:
>>> > >
>>> > > > Trying out #2, I got:
>>> > > >
>>> > > >
>>> > > >
>>> > >
>>> >
>>> testExcludeAllFromMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
>>> > > >  Time elapsed: 35.746 sec  <<< FAILURE!
>>> > > > java.lang.AssertionError
>>> > > >         at org.junit.Assert.fail(Assert.java:86)
>>> > > >         at org.junit.Assert.assertTrue(Assert.java:41)
>>> > > >         at org.junit.Assert.assertTrue(Assert.java:52)
>>> > > >         at
>>> > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
>>> > > >         at
>>> > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeAllFromMinorCompaction(TestHFileOutputFormat.java:713)
>>> > > >
>>> > > > Arun Murthy got to know this defect. Another 2.0 release is coming
>>> soon
>>> > > > (next week, hopefully).
>>> > > >
>>> > > > I plan to wait for the new release.
>>> > > >
>>> > > > Thanks
>>> > > >
>>> > > > On Fri, Feb 22, 2013 at 11:39 AM, Ted Yu <[EMAIL PROTECTED]>
>>> wrote:
>>> > > >
>>> > > > > The root cause was the removal of capacity-scheduler.xml<
>>> > > >
>>> > >
>>> >
>>> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml
>>> > > >
>>> > > > from
>>> > > > > 2.0.3-alpha maven artifacts.
>>> > > > > This config file is needed to start CapacityScheduler which is
>>> the
>>> > > > default.
>>> > > > >
>>> > > > > After talking to Sid, I got two suggestions:
>>> > > > > 1. embed minimal capacity-scheduler.xml<
>>> > > >
>>> > >
>>> >
>>> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml
>>> > > >
>>> > > > in
>>> > > > > our src/test/resources
>>> > > > > 2. use FiFo scheduler instead which doesn't require presence of
>>> > config
>>> > > > file
>>> > > > >
>>> > > > > I think #2, though temporary fix, is easy to change once a
>>> future 2.0
>>> > > > > release addresses this issue.
>>> > > > >
>>> > > > > Once consensus is reached, I will attach another patch to
>>> HBASE-7904
>>> > > > >
>>> > > > > Cheers
+
Andrew Purtell 2013-03-05, 00:10
+
Enis Söztutar 2013-02-22, 19:38
+
ramkrishna vasudevan 2013-02-22, 17:56
+
Jean-Marc Spaggiari 2013-02-22, 17:53