Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - HBASE and HIVE Integration


+
vijay shinde 2012-07-25, 08:45
+
Bejoy KS 2012-07-25, 09:30
+
vijay shinde 2012-07-25, 13:28
+
kulkarni.swarnim@...) 2012-07-25, 13:32
+
Bejoy Ks 2012-07-25, 13:47
+
vijay shinde 2012-07-26, 00:58
Copy link to this message
-
Re: HBASE and HIVE Integration
Vijay 2012-07-26, 11:30


Sent from my iPhone

On Jul 25, 2012, at 8:58 PM, vijay shinde <[EMAIL PROTECTED]> wrote:

> Hi Bejoy,
>  
> I made some changes as per your suggetion.
>  
> Here is the error from the http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207251858_0004 Job:
>  
> Error: java.lang.ClassNotFoundException: org.apache.zookeeper.KeeperException
>  at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>  at sun.misc.Launcher$
>  
> I went ahead and updated the hadoop-env.sh file and set the class path for hbase and zookeeper as follows:
> # Extra Java CLASSPATH elements.  Optional.
> export HADOOP_CLASSPATH="/usr/lib/hive/lib/hbase-0.92.jar:/usr/lib/hive/lib/zookeeper-3.4.3.jar:$HADOOP_CLASSPATH"
>  
> Here is snippet of hive-site.xml file
>  
> <name>hive.aux.jars.path</name>
> <value>file:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u4.jar,file:///usr/lib/hive/lib/hbase-0.92.jar,file:///usr/lib/hive/lib/zookeeper-3.4.3.jar,file:///usr/lib/hive/lib/hive-contrib-0.7.1-cdh3u4.jar</value>
> </property>
> <name>hbase.zookeeper.quorum</name>
> <value>localhost</value>
> </property>
>  
> Error message while executing hive query
>  
> [root@localhost hive]# ./bin/hive
> Hive history file=/tmp/root/hive_job_log_root_201207252044_1993919630.txt
> hive> INSERT OVERWRITE TABLE hive_hbasetable_k SELECT * FROM pokes where foo=98;
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> Starting Job = job_201207251858_0004, Tracking URL = http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207251858_0004
> Kill Command = /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=0.0.0.0:8021 -kill job_201207251858_0004
> 2012-07-25 20:46:38,207 Stage-0 map = 0%, reduce = 0%
> 2012-07-25 20:47:35,920 Stage-0 map = 100%, reduce = 100%
> Ended Job = job_201207251858_0004 with errors
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> hive>
>
> I am lost need help badly!!!!!!!!!!!!!
>  
> Vijay
>
>
>  
> On Wed, Jul 25, 2012 at 9:47 AM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
> Hi Vijay
>
> You have provided the hbase master directly. (It is fine for single node hbase installation). But still can you try providing the zookeeper quorum instead.
>
> If that doesn't work as well , please post the error log from the mapreduce tasks?
>
> Just go the jobtracker page and drill down on the corresponding job to get the failed tasks. From each failed tasks you can get the error logs.
> http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201207250246_0005
>
>
> Regards
> Bejoy KS
>
> From: vijay shinde <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> Sent: Wednesday, July 25, 2012 6:58 PM
> Subject: Re: HBASE and HIVE Integration
>
> Hi Bejoy,
>  
> Thanks for quick reply. Here are some additional details
>  
> Cloudera Version - CDH3U4
>  
> hive-site.xml
> <property>
> <name>hive.aux.jars.path</name>
> <value>file:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u2.jar,file:///usr/lib/hive/lib/hbase-0.90.4-cdh3u2.jar,file:///usr/lib/hive/lib/zookeeper-3.3.1.jar,file:///usr/lib/hive/lib/hive-contrib-0.7.1-cdh3u2.jar</value>
> </property>
>
> Execution Log
>  
> 1. start zookeeper
> [root@localhost zookeeper]# ./bin/zkServer.sh start
>  
> 2. start hbase
>  
> 3. start hive. I am setting hive jars in hive-site.xml
>  
> ./bin/hive -hiveconf hbase.master=127.0.1.1:60010
>  
> 4. Create new HBase table which is to be managed by Hive
>  
> CREATE TABLE hive_hbasetable_k(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "hivehbasek");
>  
> 5. Create a logical table pokes in Hive
> CREATE TABLE pokes (foo INT, bar STRING);
+
Hezhiqiang 2012-07-26, 03:36
+
Bejoy Ks 2012-07-26, 13:55
+
vijay shinde 2012-07-26, 18:54
+
Bejoy Ks 2012-07-26, 19:39
+
vijay shinde 2012-07-26, 20:44