Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> RE: Getting HBaseStorage() to work in Pig


+
Botelho, Andrew 2013-08-23, 20:57
Copy link to this message
-
Re: Getting HBaseStorage() to work in Pig
You are here running multiple UNIX commands and the end result or the end
command is to run hbase-<YOUR VERSION>.jar using hadoop's *jar* command. So
basically you add HBase jars to the classpath of your Hadoop environment
and then execute hbase tools using hadoop. If you get the message as
specified in the doc then it means that you have successfully added Hbase
libs to your Hadoop setup.

First you are setting your HADOOP_CLASSPATH by assigning it the classpath
of your HBase libs by executing the following command.
*`${HBASE_HOME}/bin/hbase classpath` *

Any command within back ticks in unix shell, is executed and its output is
being assigned to your HADOOP_CLASSPATH. Note that the 'classpath' argument
or command to the /bin/hbase executable returns the classpath of your HBase
setup.

Then
*${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/hbase-VERSION.jar*

you are running the hadoop executable with the 'jar' command (
http://hadoop.apache.org/docs/r1.0.4/commands_manual.html#jar) The 'jar'
command takes a jar as argument and here you are passing it your Hbase main
lib.jar to run. Note that there are 2 parts here. The main command is:
*${HADOOP_HOME}/bin/hadoop jar *
*
*
and the argument, the jar file is:
*${HBASE_HOME}/hbase-VERSION.jar*

${STRING} is the convention to refer to properties' holders/variables.

If you get the message from the docs, it means you have set your Hbase jars
to your Hadoop classpath correcty.

Regards,
Shahab
On Fri, Aug 23, 2013 at 4:57 PM, Botelho, Andrew <[EMAIL PROTECTED]>wrote:

> Could you explain what is going on here:****
>
> HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath`
> ${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/hbase-VERSION.jar****
>
> ** **
>
> I’m not a Unix expert by any means.****
>
> How can I use this to enable HBaseStorage() in Pig?****
>
> ** **
>
> Thanks,****
>
> ** **
>
> Andrew****
>
> ** **
>
> *From:* Ted Yu [mailto:[EMAIL PROTECTED]]
> *Sent:* Friday, August 23, 2013 4:50 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Getting HBaseStorage() to work in Pig****
>
> ** **
>
> Please look at the example in 15.1.1 under
> http://hbase.apache.org/book.html#tools****
>
> ** **
>
> On Fri, Aug 23, 2013 at 1:41 PM, Botelho, Andrew <[EMAIL PROTECTED]>
> wrote:****
>
> I am trying to use the function HBaseStorage() in my Pig code in order to
> load an HBase table into Pig.****
>
>  ****
>
> When I run my code, I get this error:****
>
>  ****
>
> ERROR 2998: Unhandled internal error.
> org/apache/hadoop/hbase/filter/WritableByteArrayComparable****
>
>  ****
>
>  ****
>
> I believe the PIG_CLASSPATH needs to be extended to include the classpath
> for loading HBase, but I am not sure how to do this.  I've tried several
> export commands in the unix shell to change the PIG_CLASSPATH, but nothing
> seems to be working.****
>
>  ****
>
> Any advice would be much appreciated.****
>
>  ****
>
> Thanks,****
>
>  ****
>
> Andrew Botelho****
>
>  ****
>
> ** **
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB