Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> HBASE and HIVE Integration


Copy link to this message
-
Re: HBASE and HIVE Integration


Sent from my iPhone

On Jul 25, 2012, at 8:58 PM, vijay shinde <[EMAIL PROTECTED]> wrote:

> Hi Bejoy,
>  
> I made some changes as per your suggetion.
>  
> Here is the error from the http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207251858_0004 Job:
>  
> Error: java.lang.ClassNotFoundException: org.apache.zookeeper.KeeperException
>  at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>  at sun.misc.Launcher$
>  
> I went ahead and updated the hadoop-env.sh file and set the class path for hbase and zookeeper as follows:
> # Extra Java CLASSPATH elements.  Optional.
> export HADOOP_CLASSPATH="/usr/lib/hive/lib/hbase-0.92.jar:/usr/lib/hive/lib/zookeeper-3.4.3.jar:$HADOOP_CLASSPATH"
>  
> Here is snippet of hive-site.xml file
>  
> <name>hive.aux.jars.path</name>
> <value>file:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u4.jar,file:///usr/lib/hive/lib/hbase-0.92.jar,file:///usr/lib/hive/lib/zookeeper-3.4.3.jar,file:///usr/lib/hive/lib/hive-contrib-0.7.1-cdh3u4.jar</value>
> </property>
> <name>hbase.zookeeper.quorum</name>
> <value>localhost</value>
> </property>
>  
> Error message while executing hive query
>  
> [root@localhost hive]# ./bin/hive
> Hive history file=/tmp/root/hive_job_log_root_201207252044_1993919630.txt
> hive> INSERT OVERWRITE TABLE hive_hbasetable_k SELECT * FROM pokes where foo=98;
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_201207251858_0004, Tracking URL = http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207251858_0004
> Kill Command = /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=0.0.0.0:8021 -kill job_201207251858_0004
> 2012-07-25 20:46:38,207 Stage-0 map = 0%, reduce = 0%
> 2012-07-25 20:47:35,920 Stage-0 map = 100%, reduce = 100%
> Ended Job = job_201207251858_0004 with errors
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> hive>
>
> I am lost need help badly!!!!!!!!!!!!!
>  
> Vijay
>
>
>  
> On Wed, Jul 25, 2012 at 9:47 AM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
> Hi Vijay
>
> You have provided the hbase master directly. (It is fine for single node hbase installation). But still can you try providing the zookeeper quorum instead.
>
> If that doesn't work as well , please post the error log from the mapreduce tasks?
>
> Just go the jobtracker page and drill down on the corresponding job to get the failed tasks. From each failed tasks you can get the error logs.
> http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207250246_0005
>
>
> Regards
> Bejoy KS
>
> From: vijay shinde <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> Sent: Wednesday, July 25, 2012 6:58 PM
> Subject: Re: HBASE and HIVE Integration
>
> Hi Bejoy,
>  
> Thanks for quick reply. Here are some additional details
>  
> Cloudera Version - CDH3U4
>  
> hive-site.xml
> <property>
> <name>hive.aux.jars.path</name>
> <value>file:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u2.jar,file:///usr/lib/hive/lib/hbase-0.90.4-cdh3u2.jar,file:///usr/lib/hive/lib/zookeeper-3.3.1.jar,file:///usr/lib/hive/lib/hive-contrib-0.7.1-cdh3u2.jar</value>
> </property>
>
> Execution Log
>  
> 1. start zookeeper
> [root@localhost zookeeper]# ./bin/zkServer.sh start
>  
> 2. start hbase
>  
> 3. start hive. I am setting hive jars in hive-site.xml
>  
> ./bin/hive -hiveconf hbase.master=127.0.1.1:60010
>  
> 4. Create new HBase table which is to be managed by Hive
>  
> CREATE TABLE hive_hbasetable_k(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "hivehbasek");
>  
> 5. Create a logical table pokes in Hive
> CREATE TABLE pokes (foo INT, bar STRING);
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB