Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # dev >> Hadoop 2.0 Support for Accumulo 1.4 Branch


Copy link to this message
-
Re: Hadoop 2.0 Support for Accumulo 1.4 Branch
On Mon, Oct 14, 2013 at 9:24 PM, Mike Drob <[EMAIL PROTECTED]> wrote:

>
> > 3) Test for correctness on given versions, with >= 5 node cluster
> >
> > * Unit Tests
> > * Functional Tests
> > * 24hr continuous + verification
> > * 24hr continuous + verification + agitation
> > * 24hr random walk
> > * 24hr random walk + agitation
> >
> > Keith mentioned running these against a CDH4 cluster, but I presume that
> > since Apache Releases are our stated compatibilities it would actually be
> > against whatever versions we list. Based on #1 and #2 above, I would
> expect
> > that to be Apache Hadoop 0.20.203.0 and Apache Hadoop 2.0.4-alpha.
> >
> Hadoop 2 introduces some neat new things like NN HA, which I think it
> might be worthwhile to test with. At that level it might be more of a
> verification of the Hadoop code, but I'd like to be comfortable that our
> DFS Clients switch correctly. This is in addition to the standard release
> suite that we run. [1]
>
> [1]: http://accumulo.apache.org/governance/releasing.html#testing
>
>
>

Just to confirm, the change from Keith's request is

* 72hr continuous + agitation + cluster running
* Something to test that HA NN failover doesn't take out Accumulo

Would the latter be addressed by an additional functional test? or would it
need to be some kind of addition to the agitation?

>
> Having run the binary packaging for 1.4.4, I can tell you that it is not in
> great shape. Christopher cleaned up a lot of the issues in the 1.5 line, so
> I didn't bother spending a ton of time on them here, but I think RPM and
> DEB are both broken. It would be nice to be able to specify a Hadoop 2
> version for compilation, similar to what happens in the newer code base,
> which could be back ported, I suppose. 4b seems easier.
>
>

I think this means you're +0 on 4b?

>  =application
> >
> > There will be many back-ported patches. Not much active development
> happens
> > on 1.4.x now, but I presume this should still all go onto a feature
> branch?
> >
> > Is the community preference that eventually all the changes become a
> single
> > commit (or one-per-subtask if there are multiple jiras) on the active 1.4
> > development branch, or that the original patches remain broken out?
> >
> Not sure what you mean by this.
>
>
It's the difference between the 1.4.x branch having all the commits that
are backported from 1.5.x vs just having squashed ones. The former
maintains more of the original authorship and ties to original jiras. The
latter has less noise.

--
Sean
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB