Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Re: Pig + Hbase integration


Copy link to this message
-
Re: Pig + Hbase integration
On Thu, Oct 25, 2012 at 7:44 AM, Manu S <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I am using Pig-0.10.0 & hbase-0.94.2.
>
> I am trying to store the processed output to Hbase cluster using pig
> script.
>
> I registered the required .jar and set the mapreduce and zookeeper
> parameters within the script itself.
>
> *# cat input.pig*
> register jar/hbase-0.94.2.jar;
> register jar/zookeeper-3.4.3.jar;
> register jar/protobuf-java-2.4.0a.jar;
> register jar/guava-11.0.2.jar;
> register jar/pig-0.10.0.jar;
>
> set fs.default.name hdfs://namenode:8020;
> set mapred.job.tracker namenode:8021;
> set hbase.cluster.distributed true;
> set hbase.zookeeper.quorum namenode;
> set hbase.master namenode:60000;
> set hbase.zookeeper.property.clientPort 2181;
> *
> *
> *raw_data = LOAD 'sample_data.csv' USING PigStorage( ',' ) AS (
>  listing_id: chararray,fname: chararray,lname: chararray );*
> *
> *
> *STORE raw_data INTO 'hbase://inputcsv' USING
> org.apache.pig.backend.hadoop.hbase.HBaseStorage ('info:fname info:lname');*
>
> When I execute the script I am getting this error
>
> *# pig input.pig*
> *2012-10-25 19:55:08,331 [main] INFO  org.apache.pig.Main - Apache Pig
> version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12*
> *2012-10-25 19:55:08,332 [main] INFO  org.apache.pig.Main - Logging error
> messages to: /export/home/hadoop/devel/pig/pig_1351175108325.log*
> *2012-10-25 19:55:08,944 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: hdfs://sangamt4:8020*
> *2012-10-25 19:55:09,172 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to map-reduce job tracker at: sangamt4:8021*
> *2012-10-25 19:55:10,021 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 2998: Unhandled internal error.
> org/apache/hadoop/hbase/filter/WritableByteArrayComparable*
> *Details at logfile: /export/home/hadoop/devel/pig/pig_1351175108325.log*

And what are the details like in that log? Is it a classpath problem?

J-D
+
Manu S 2012-10-30, 05:27
+
Krishna Kalyan 2014-09-27, 03:32
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB