Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Exception (possibly) due to type casting between bytearray to charray in HBaseStorage


Copy link to this message
-
Exception (possibly) due to type casting between bytearray to charray in HBaseStorage
Hello,

When loading data from a HBase table in Pig, using HBaseStorage, if I
specify the type of the fields as chararray, I get an exception:

2013-05-29 16:18:56,557 INFO
org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
truncater with mapRetainSize=-1 and reduceRetainSize=-1
2013-05-29 16:18:56,560 FATAL org.apache.hadoop.mapred.Child: Error
running child : java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/mapreduce/TableInputFormat
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)


events = LOAD 'hbase://events'
USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:field1
info:field2 info:field3')
AS (field1: chararray, field2:chararray, field3:chararray);
It seems to work if I use bytearray instead of chararray (or no type at
all.)
AS (field1: bytearray, field2:bytearray, field3:bytearray);

or

AS (field1, field2, field3);

My question is that why do we get the ClassNotFoundException for
TableInpuFormat which is kind of misleading? Plus, is it even type casting
really the issue? Wouldn't automatic byte-to-char conversion happen?

This issue has been discussed earlier also but it doesn't seem any
conclusion was reached. Plus so far, I have not seen any problem regarding
HBase libs anywhere:
http://search-hadoop.com/m/KUU6m1oZMEi1&subj=Re+Unable+to+typecast+fields+loaded+from+HBase

Thanks.

Regards,
Shahab
+
Praveen Bysani 2013-05-30, 03:26
+
Shahab Yunus 2013-05-30, 12:14
+
Praveen Bysani 2013-05-31, 08:17
+
Shahab Yunus 2013-05-31, 11:19
+
shashwat shriparv 2013-05-31, 11:44
+
Shahab Yunus 2013-05-31, 14:46