Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - coprocessorExec got stucked with generic type


Copy link to this message
-
Re: coprocessorExec got stucked with generic type
Pavel Hančar 2013-07-02, 09:55
oops - I forgot to answer. The NameAndDistance doesn't implement
org.apache.hadoop.io.Writable, so thanks Gary, it looks like a good
alternative. But I stayed with SerializationUtils, because it looks simpler
than to implement the read and write methods.
 Pavel

2013/6/25 Kim Chew <[EMAIL PROTECTED]>

> I think I have run into a similar situation like Pavel.
>
> My method returns Map<Long, ArrayList<Foo>>, where Foo is
>
> public class Foo implements Writable {
>      String something;
>      long    counter1;
>      long    counter2;
>
>      blah........
> };
>
> And I got the following exception when I called my coprocessor method from
> a client,
>
> java.io.NotSerializableException: Foo
>     at
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
>     at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
>     at java.util.ArrayList.writeObject(ArrayList.java:570)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     at java.lang.reflect.Method.invoke(Method.java:597)
>     at
> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
>     at
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469)
>     at
>
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
>     at
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
>     at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
>     at java.util.HashMap.writeObject(HashMap.java:1001)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     at java.lang.reflect.Method.invoke(Method.java:597)
>     at
> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
>     at
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469)
>     at
>
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
>     at
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
>     at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
>     at
>
> org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWritable.java:540)
>     at
>
> org.apache.hadoop.hbase.client.coprocessor.ExecResult.write(ExecResult.java:76)
>     at
>
> org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWritable.java:525)
>     at
>
> org.apache.hadoop.hbase.io.HbaseObjectWritable.write(HbaseObjectWritable.java:335)
>     at
>
> org.apache.hadoop.hbase.ipc.HBaseServer$Call.setResponse(HBaseServer.java:365)
>     at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1442)
>
> What I do not understand is, how does the predicate at line 526 in
> HbaseObjectWritable is true if Foo does not implement Serializable?
>
>  } else if (Serializable.class.isAssignableFrom(declClass))
>
> BTW, making Foo serializable worked.
>
> Regards,
> Kim
>
>
> On Tue, Jun 11, 2013 at 9:48 AM, Gary Helmling <[EMAIL PROTECTED]>
> wrote:
>
> > Does your NameAndDistance class implement org.apache.hadoop.io.Writable?
> >  If so, it _should_ be serialized correctly.  There was a past issue
> > handling generic types in coprocessor endpoints, but that was fixed way
> > back (long before 0.94.2).  So, as far as I know, this should all be
> > working, assuming that NameAndDistance can be serialized.
> >
> >
> > On Mon, Jun 10, 2013 at 9:36 AM, Pavel Hančar <[EMAIL PROTECTED]
> > >wrote:
> >
> > >  It's org.apache.commons.lang.SerializationUtils
> > > I have it in hbase-0.94.2-cdh4.2.1/lib/commons-lang-2.5.jar
> > >  Pavel
> > >
> > >
> > > 2013/6/10 Ted Yu <[EMAIL PROTECTED]>