Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Hadoop 0.22/HBase 0.92 package repos are now available


Copy link to this message
-
Re: Hadoop 0.22/HBase 0.92 package repos are now available
On Wed, Sep 28, 2011 at 6:28 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> On Wed, Sep 28, 2011 at 11:54 AM,  <[EMAIL PROTECTED]> wrote:
>> This is great progress folks !!!
>
> Thanks. Actually, for all those who care about .23 I've created a hadoop-0.23
> branch in Bigtop and hooked up 3 additional Jenkins jobs:
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-centos5/
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-ubuntu-lucid/
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/
>
> The good news is that HDFS and common are ok, can be packaged and
> uses. The bad news for .23 are way more plentiful though:
>     1. HBase now breaks with the profile that is supposed to be compatible
>     with .23. Perhaps this is one of the issues that Todd has mentioned
>     earlier, but I'm not sure.
>
>     2. MR2 (or more precise YARN) seem to be in a pretty weird shape.
>     First of all, I can't really build it the same way that the rest of the
>     Hadoop 0.23 builds (which is what specified in top-level BUILDING.txt).
>     Somehow it requires assembly:single goal to be executed explicitly.
>     Worse yet, it seems to be following a different binary layout compared to
>     HDFS and common (conf vs. etc/hadoop, etc.). Also, I'm not sure how
>     to make natives bits of pipes/utils compile short of manual execution
>     of ant.
>
>     3. fuse-dfs is now an orphan. Its not hooked to the build system and seems
>     to be in limbo: https://issues.apache.org/jira/browse/HDFS-1757
>
> I would really, really, really appreciate if somebody can help triaging these
> issues or at least would be nice enough to point me to existing
> JIRAs/discussions.

Regarding 2. At the moment you can build MR following the instructions
in the INSTALL file. In fact the nightly build set up by Giri and
Mahadev builds a MR tarball
(https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-MapReduce-trunk).
There are some JIRAs to remove the inconsistencies with the rest of
the build, such as MAPREDUCE-3024, MAPREDUCE-3014, HADOOP-7590,
HADOOP-7642. I hope to work on some of these soon, but any help is
always appreciated.

Cheers,
Tom

>
> My goal here is to get us to a point where we can start executing
> automated integration testing against a fully distributed cluster and
> start publishing these results on the same Jenkins to monitor the
> health and progress of .22/.23 and corresponding Bigtop stacks.
>
> Thanks,
> Roman.
>