Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Re: Hadoop not working after replacing hadoop-core.jar with hadoop-core-append.jar


Copy link to this message
-
Re: Hadoop not working after replacing hadoop-core.jar with hadoop-core-append.jar
I have been looking at

http://hbase.apache.org/notsoquick.html#hadoop

which does NOT have that citation.  So I never saw that before now.  It is
indeed helpful.  But: must we really spend hours on flaky tests while
building?  Also, it would comfort noobs like me if there were a bit of
explanation relating to the hadoop build instructions, which confusingly
seem to sometimes build native binaries and sometimes not.

Note: the remark that

"You have to replace it if you are running on an hadoop that is other
than an exact match to the jar we ship with"

suggests one could (if "exact match") go down the very path that we are
trying to discourage (even when "exact match").  This is what I tried, and
was then told is not reliable, then was told might work, but nobody was
willing to tell me whether or how I could be sure it would work.  So my
suggestion is to be unequivocal about it: when running distributed, always
build your own Hadoop and put its -core JAR into your HBase installation
(or use Cloudera, which has done this for you).  Also: explicitly explain
how the file has to be named (there is a strict naming requirement so that
the launching scripts work, right?).

Regards,
Mike Spreitzer
From:   stack <[EMAIL PROTECTED]>
To:     Mike Spreitzer/Watson/IBM@IBMUS
Cc:     [EMAIL PROTECTED]
Date:   06/06/2011 10:58 PM
Subject:        Re: Hadoop not working after replacing hadoop-core.jar
with hadoop-core-append.jar

On Mon, Jun 6, 2011 at 6:49 PM, Mike Spreitzer <[EMAIL PROTECTED]>
wrote:
> Where is that citation of Michael Noll's nicely detailed instruction on
how
> to build the append branch?
>

See Section 1.3.1.2 here
http://hbase.apache.org/book/notsoquick.html#requirements.  Look for
"Michael Noll has written a detailed blog, Building an Hadoop 0.20.x
version for HBase 0.90.2, on how to build an Hadoop from
branch-0.20-append. Recommended."

> Why does hbase include a hadoop-core.jar?  The instructions say I should
> replace it, so why am I given it in the first place?
>

You have to replace it if you are running on an hadoop that is other
than an exact match to the jar we ship with (If you are doing
standalone mode or if you are running unit tests, the jar is needed
since we have a bunch of Hadoop dependencies from our Configuration to
UI to MapReduce to Connection to HDFS etc.)

Again, I apologize for the fact that this is less-than smooth sailing.
 HBase project is in a bit of an awkward spot.  We're trying to put
the best face on it.  If you have any suggestions for how best we
might this, we are all ears.

Yours,
St.Ack

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB