Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - Update on hadoop-0.23


Copy link to this message
-
Re: Update on hadoop-0.23
Roman Shaposhnik 2011-09-26, 22:15
Hi Arun!

Great news! Hopefuly you wouldn't mind answering some of the questions below...

On Mon, Sep 26, 2011 at 2:07 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> NextGen MapReduce (aka MRv2, aka YARN) is coming along great:
> # We are happy to report we've done extensive scale testing to confirm stability
>  - Sort/GridMixv3 etc. at ~350nodes
>  - Scale testing with simulated clusters of ~1500 nodes
> # Functional tests for all of MapReduce functionality
> # Pig  (0.9 & 0.9.1) working with NextGen MapReduce

Is there a *released* version of Pig that compiles cleanly against .23
snapshots?
Same question for Hive.

> We are about to finish performance certification for both HDFS & MapReduce in the next
> couple of weeks too, after which we start integration tests with HBase, Hive, Oozie etc.

I'm curious -- what are these integrations tests? Can I take a look at
them? I would
be really nice if we can levarage those via Bigtop infrastructure. Currently we
have a certain # of integration tests in Bigtop that we're running
against a fully
deployed stack, but it would be quite nice to have extra coverage.

> Given where we are I'm confident we can have a strong hadoop-0.23.0 release
> by late October. The current plan is to deploy to alpha clusters in November. Citius, Altius, Fortius! :)

Could you, please, elaborate on what will be part of that deployment?
Which versions
of Pig, Hive, HBase, Oozie and Mahout are you targeting?

Thanks,
Roman.