Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Update on hadoop-0.23


Copy link to this message
-
Re: Update on hadoop-0.23
Hi Arun!

Great news! Hopefuly you wouldn't mind answering some of the questions below...

On Mon, Sep 26, 2011 at 2:07 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> NextGen MapReduce (aka MRv2, aka YARN) is coming along great:
> # We are happy to report we've done extensive scale testing to confirm stability
>  - Sort/GridMixv3 etc. at ~350nodes
>  - Scale testing with simulated clusters of ~1500 nodes
> # Functional tests for all of MapReduce functionality
> # Pig  (0.9 & 0.9.1) working with NextGen MapReduce

Is there a *released* version of Pig that compiles cleanly against .23
snapshots?
Same question for Hive.

> We are about to finish performance certification for both HDFS & MapReduce in the next
> couple of weeks too, after which we start integration tests with HBase, Hive, Oozie etc.

I'm curious -- what are these integrations tests? Can I take a look at
them? I would
be really nice if we can levarage those via Bigtop infrastructure. Currently we
have a certain # of integration tests in Bigtop that we're running
against a fully
deployed stack, but it would be quite nice to have extra coverage.

> Given where we are I'm confident we can have a strong hadoop-0.23.0 release
> by late October. The current plan is to deploy to alpha clusters in November. Citius, Altius, Fortius! :)

Could you, please, elaborate on what will be part of that deployment?
Which versions
of Pig, Hive, HBase, Oozie and Mahout are you targeting?

Thanks,
Roman.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB