Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # dev >> Hadoop 2.0 Support for Accumulo 1.4 Branch


+
Joey Echeverria 2013-07-26, 15:02
+
Eric Newton 2013-07-26, 15:25
+
Josh Elser 2013-07-26, 15:36
+
Keith Turner 2013-07-26, 15:45
+
Joey Echeverria 2013-07-26, 16:24
+
Keith Turner 2013-07-26, 17:20
+
Joey Echeverria 2013-07-26, 18:33
+
Billie Rinaldi 2013-07-26, 19:02
+
dlmarion@... 2013-07-26, 19:34
+
Joey Echeverria 2013-07-29, 17:23
+
Dave Marion 2013-08-01, 23:33
+
Joey Echeverria 2013-08-02, 18:22
+
Christopher 2013-08-02, 18:31
+
Joey Echeverria 2013-08-02, 18:37
+
Mike Drob 2013-08-02, 19:58
+
Joey Echeverria 2013-08-02, 20:03
+
Billie Rinaldi 2013-08-04, 22:54
Copy link to this message
-
Re: Hadoop 2.0 Support for Accumulo 1.4 Branch
On Fri, Jul 26, 2013 at 2:33 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:

> > If these patches are going to be included with 1.4.4 or 1.4.5, I would
> like
> > to see the following test run using CDH4 on at least a 5 node cluster.
> >  More nodes would be better.
> >
> >   * unit test
> >   * Functional test
> >   * 24 hr Continuous ingest + verification
> >   * 24 hr Continuous ingest + verification + agitation
> >   * 24 hr Random walk
> >   * 24 hr Random walk + agitation
> >
> > I may be able to assist with this, but I can not make any promises.
>
> Sure thing. Is there already a write-up on running this full battery
> of tests? I have a 10 node cluster that I can use for this.
>

There are some instructions.

test/system/continuous/README
test/system/randomwalk/README

Continuous ingest has a lot of options.  For release testing we do
something like the following.

  #configure may need to adjust max mappers and max reducers to make map
reduce job run faster
  start-ingest.sh
  start-walker.sh
  #sleep 24hr
  stop-ingest.sh
  stop-walker.sh
  run-verify.sh

There continuous dir has scripts for starting and stopping the agitator.
 We also use this script to agitate while running random walk test.

For random walk we use the All.xml graph, configure it to log errors to
NFS, and run a walker on each node.  We look in NFS for walkers that died
or got stuck.  The random walk framework will log a message if a node in
the graph gets stuck.  It will also log a message when it gets unstuck.
>
> > Great.  I think this would be a good patch for 1.4.   I assume that if a
> > user stays with Hadoop 1 there are no dependency changes?
>
> Yup. It works the same way as 1.5 where all of the dependency changes
> are in a Hadoop 2.0 profile.
>
> -Joey
>
+
Sean Busbey 2013-10-14, 16:55
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB