Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Hadoop 0.22/HBase 0.92 package repos are now available


Copy link to this message
-
Re: Hadoop 0.22/HBase 0.92 package repos are now available
On Wed, Sep 28, 2011 at 6:28 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> On Wed, Sep 28, 2011 at 11:54 AM,  <[EMAIL PROTECTED]> wrote:
>> This is great progress folks !!!
>
> Thanks. Actually, for all those who care about .23 I've created a hadoop-0.23
> branch in Bigtop and hooked up 3 additional Jenkins jobs:
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-centos5/
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-ubuntu-lucid/
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/
>
> The good news is that HDFS and common are ok, can be packaged and
> uses. The bad news for .23 are way more plentiful though:
>     1. HBase now breaks with the profile that is supposed to be compatible
>     with .23. Perhaps this is one of the issues that Todd has mentioned
>     earlier, but I'm not sure.
>
>     2. MR2 (or more precise YARN) seem to be in a pretty weird shape.
>     First of all, I can't really build it the same way that the rest of the
>     Hadoop 0.23 builds (which is what specified in top-level BUILDING.txt).
>     Somehow it requires assembly:single goal to be executed explicitly.
>     Worse yet, it seems to be following a different binary layout compared to
>     HDFS and common (conf vs. etc/hadoop, etc.). Also, I'm not sure how
>     to make natives bits of pipes/utils compile short of manual execution
>     of ant.
>
>     3. fuse-dfs is now an orphan. Its not hooked to the build system and seems
>     to be in limbo: https://issues.apache.org/jira/browse/HDFS-1757
>
> I would really, really, really appreciate if somebody can help triaging these
> issues or at least would be nice enough to point me to existing
> JIRAs/discussions.

Regarding 2. At the moment you can build MR following the instructions
in the INSTALL file. In fact the nightly build set up by Giri and
Mahadev builds a MR tarball
(https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-MapReduce-trunk).
There are some JIRAs to remove the inconsistencies with the rest of
the build, such as MAPREDUCE-3024, MAPREDUCE-3014, HADOOP-7590,
HADOOP-7642. I hope to work on some of these soon, but any help is
always appreciated.

Cheers,
Tom

>
> My goal here is to get us to a point where we can start executing
> automated integration testing against a fully distributed cluster and
> start publishing these results on the same Jenkins to monitor the
> health and progress of .22/.23 and corresponding Bigtop stacks.
>
> Thanks,
> Roman.
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB