Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Exception (possibly) due to type casting between bytearray to charray in HBaseStorage


+
Shahab Yunus 2013-05-29, 22:24
+
Praveen Bysani 2013-05-30, 03:26
+
Shahab Yunus 2013-05-30, 12:14
Copy link to this message
-
Re: Exception (possibly) due to type casting between bytearray to charray in HBaseStorage
Yes, it could be. Try registering these jar files

/opt/cloudera/parcels/CDH-<version>/lib/hbase/hbase-<version>-security.jar
/opt/cloudera/parcels/CDH-<version>/lib/zookeeper/zookeeper-<version>.jar

On 30 May 2013 20:14, Shahab Yunus <[EMAIL PROTECTED]> wrote:

> I am not explicitly registering any of these jars in the script. The
> cluster was setup through standard Cloudera installation (4.2.0). Should I?
> Is that the issue?
>
> Regards,
> Shahab
>
>
> On Wed, May 29, 2013 at 11:26 PM, Praveen Bysani <[EMAIL PROTECTED]
> >wrote:
>
> > Hi,
> >
> > Are you registering the hbase and zookeepr jar files in your pig script ?
> >
> > On 30 May 2013 06:24, Shahab Yunus <[EMAIL PROTECTED]> wrote:
> >
> > > Hello,
> > >
> > > When loading data from a HBase table in Pig, using HBaseStorage, if I
> > > specify the type of the fields as chararray, I get an exception:
> > >
> > > 2013-05-29 16:18:56,557 INFO
> > > org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
> > > truncater with mapRetainSize=-1 and reduceRetainSize=-1
> > > 2013-05-29 16:18:56,560 FATAL org.apache.hadoop.mapred.Child: Error
> > > running child : java.lang.NoClassDefFoundError:
> > > org/apache/hadoop/hbase/mapreduce/TableInputFormat
> > >         at java.lang.ClassLoader.defineClass1(Native Method)
> > >         at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> > >         at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> > >         at
> > > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> > >         at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> > >         at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> > >         at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> > >         at java.security.AccessController.doPrivileged(Native Method)
> > >         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > >         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > >         at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > >         at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > >         at java.lang.Class.forName0(Native Method)
> > >         at java.lang.Class.forName(Class.java:247)
> > >
> > >
> > > events = LOAD 'hbase://events'
> > > USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:field1
> > > info:field2 info:field3')
> > > AS (field1: chararray, field2:chararray, field3:chararray);
> > > It seems to work if I use bytearray instead of chararray (or no type at
> > > all.)
> > > AS (field1: bytearray, field2:bytearray, field3:bytearray);
> > >
> > > or
> > >
> > > AS (field1, field2, field3);
> > >
> > > My question is that why do we get the ClassNotFoundException for
> > > TableInpuFormat which is kind of misleading? Plus, is it even type
> > casting
> > > really the issue? Wouldn't automatic byte-to-char conversion happen?
> > >
> > > This issue has been discussed earlier also but it doesn't seem any
> > > conclusion was reached. Plus so far, I have not seen any problem
> > regarding
> > > HBase libs anywhere:
> > >
> > >
> >
> http://search-hadoop.com/m/KUU6m1oZMEi1&subj=Re+Unable+to+typecast+fields+loaded+from+HBase
> > >
> > > Thanks.
> > >
> > > Regards,
> > > Shahab
> > >
> >
> >
> >
> > --
> > Regards,
> > Praveen Bysani
> > http://www.praveenbysani.com
> >
>

--
Regards,
Praveen Bysani
http://www.praveenbysani.com
+
Shahab Yunus 2013-05-31, 11:19
+
shashwat shriparv 2013-05-31, 11:44
+
Shahab Yunus 2013-05-31, 14:46