Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Default hadoop version for 0.94?


Copy link to this message
-
Re: Default hadoop version for 0.94?
There appear to be significant performance advantages to using Hadoop 1.2.1.
I was just debugging why the VerifyReplication test takes 190s in 0.94 vs. 39s in HBase trunk.
After some debugging it turns out the difference is due to the different default Hadoop versions (0.94 uses 1.0.4, trunk uses 1.2.1).

I have not made the change, yet, but it seems nobody should really use 1.0.x anymore (if possible, of course).
-- Lars

________________________________
 From: lars hofhansl <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; hbase-user <[EMAIL PROTECTED]>
Sent: Thursday, October 17, 2013 10:39 PM
Subject: Re: Default hadoop version for 0.94?
 

+ User list

I am planning to upgrade 0.94's default Hadoop to 1.2.1.
Please speak up now if you think this is an issue.

Thanks.

-- Lars

________________________________

From: lars hofhansl <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Thursday, October 17, 2013 7:10 PM
Subject: Re: Default hadoop version for 0.94?
Ok... I will make that so.
Of course the 1.0.x target will stay, just that the default of the sample tarballs will change to 1.2.1.

-- Lars

________________________________

From: Ted Yu <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Thursday, October 17, 2013 3:55 PM
Subject: Re: Default hadoop version for 0.94?
For hadoop-1.0 profile, currently version 1.0.4 is used.

Setting up a new build against hadoop 1.0.4 when the original build
switches to using 1.2.1 would be good.

On Thu, Oct 17, 2013 at 3:48 PM, Enis Söztutar <[EMAIL PROTECTED]> wrote:

> +1 for hadoop 1.2. However, my only concern is that we should be able to
> still build and work with 1.0.3 and 1.1.x. Maybe we should create jenkins
> build for those?
>
>
> On Thu, Oct 17, 2013 at 3:39 PM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
> > Excellent. Thanks Ted.
> >
> >
> > 2013/10/17 Ted Yu <[EMAIL PROTECTED]>
> >
> > > bq.  Will that make anything faster to have 1.2 as the default version
> > > instead of 1.0?
> > >
> > > You can get to release notes from this thread:
> > > http://search-hadoop.com/m/pFao2kKHDm1
> > >
> > > A few examples:
> > >
> > > MAPREDUCE-5368 <https://issues.apache.org/jira/browse/MAPREDUCE-5368>.
> > > Major improvement reported by zhaoyunjiong and fixed by zhaoyunjiong
> > (mrv1)
> > > *Save memory by set capacity, load factor and concurrency level for
> > > ConcurrentHashMap in TaskInProgress*
> > > *
> > > *
> > > HADOOP-8971 <https://issues.apache.org/jira/browse/HADOOP-8971>. Major
> > > improvement reported by gopalv and fixed by gopalv (util)
> > > *Backport: hadoop.util.PureJavaCrc32 cache hit-ratio is low for static
> > data
> > > (HADOOP-8926)**
> > > *
> > >
> > >
> > > On Thu, Oct 17, 2013 at 11:58 AM, Jean-Marc Spaggiari <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > > I think also we should be able to compile against v1 if required.
> Don't
> > > > think we should remove the related code yet. Might be on the roadmap,
> > but
> > > > most probably to soon to do that.  I'm not +1 because that will
> impact
> > me
> > > > ;) But can't be -1 (anyway, who cares, I'm not commiter ;)  )
> > > >
> > > > Just one question, that is this going to give us? Will that make
> > anything
> > > > faster to have 1.2 as the default version instead of 1.0?
> > > >
> > > >
> > > > 2013/10/17 Andrew Purtell <[EMAIL PROTECTED]>
> > > >
> > > > > +1, if the current default version of 1 we build against can't even
> > be
> > > > > downloaded except from the archives, that's not good.
> > > > >
> > > > >
> > > > > On Thu, Oct 17, 2013 at 11:15 AM, Nick Dimiduk <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > >
> > > > > > +1 from me as well. Keep on moving forward.
> > > > > >
> > > > > > Does this mean we can delete some old API compatibility code
> soon?
> > I
> > > <3
> > > > > > deleting code ;)
> > > > > >
> > > > > >
> > > > > > On Thu, Oct 17, 2013 at 11:09 AM, lars hofhansl <