Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - ClassNotFoundException when use hive java client of hive + hbase integration


Copy link to this message
-
Re: ClassNotFoundException when use hive java client of hive + hbase integration
Cheng Su 2012-11-06, 03:18
Mark, thank you so much for your suggestion.

Although I've already add necessary jars to my hive aux path, thus I
can execute my sql in hive CLI mode without getting any error.
But when I use a java client to access the tables through the thrift
service, I need to add these jars manually.
I execute the "ADD JAR xxxx.jar" sql and the problem is solved!

Thank you again!

On Tue, Nov 6, 2012 at 9:03 AM, Mark Grover <[EMAIL PROTECTED]> wrote:
> Cheng,
> You will have to add the appropriate HBase related jars to your class path.
>
> You can do that by running "add jar" command(s) or put it in aux_lib. See
> this thread for reference:
> http://mail-archives.apache.org/mod_mbox/hive-user/201103.mbox/%3CAANLkTingqLGKnQmiZgoi+[EMAIL PROTECTED]%3E
>
> Mark
>
>
> On Mon, Nov 5, 2012 at 6:53 AM, Cheng Su <[EMAIL PROTECTED]> wrote:
>>
>> Hi, all. I have a hive+hbase integration cluster.
>>
>> When I try to execute query through the java client of hive, sometimes
>> a ClassNotFoundException happens.
>>
>> My java code :
>>
>> final Connection conn = DriverManager.getConnection(URL);
>> final ResultSet rs = conn.executeQuery("SELECT count(*) FROM
>> test_table WHERE (source = '0' AND ur_createtime BETWEEN
>> '20121031000000' AND '20121031235959')");
>>
>> I can execute the sql:SELECT count(*) FROM test_table WHERE (source >> '0' AND ur_createtime BETWEEN '20121031000000' AND '20121031235959')
>> in hive cli mode, and get the query result, so there is no error in my
>> sql.
>>
>> The client side exception:
>>
>> Caused by: java.sql.SQLException: Query returned non-zero code: 9,
>> cause: FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>>     at
>> org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:189)
>> ... 23 more
>>
>> The server side exception(hadoop-jobtracker):
>>
>> 2012-11-05 18:55:39,443 INFO org.apache.hadoop.mapred.TaskInProgress:
>> Error from attempt_201210301133_0112_m_000000_3: java.io.IOException:
>> Cannot create an instance of InputSplit class >>
>> org.apache.hadoop.hive.hbase.HBaseSplit:org.apache.hadoop.hive.hbase.HBaseSplit
>>     at
>> org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:146)
>>     at
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
>>     at
>> org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
>>     at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:396)
>>     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:412)
>>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
>>     at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at javax.security.auth.Subject.doAs(Unknown Source)
>>     at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>>     at org.apache.hadoop.mapred.Child.main(Child.java:249)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.hadoop.hive.hbase.HBaseSplit
>>     at java.net.URLClassLoader$1.run(Unknown Source)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(Unknown Source)
>>     at java.lang.ClassLoader.loadClass(Unknown Source)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
>>     at java.lang.ClassLoader.loadClass(Unknown Source)
>>     at java.lang.Class.forName0(Native Method)
>>     at java.lang.Class.forName(Unknown Source)
>>     at
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
>>     at
>> org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:143)
>>     ... 10 more
>>
>>
>> My hive-env.sh
>>
>> export
>> HIVE_AUX_JARS_PATH=/data/install/hive-0.9.0/lib/hive-hbase-handler-0.9.0.jar,/data/install/hive-0.9.0/lib/hbase-0.92.0.jar,/data/install/hive-0.9.0/lib/zookeeper-3.4.2.jar
Regards,
Cheng Su