Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - expect specific record but get generic


+
Koert Kuipers 2013-10-21, 15:41
+
Doug Cutting 2013-10-21, 17:07
+
Koert Kuipers 2013-10-21, 20:19
+
Doug Cutting 2013-10-21, 20:49
Copy link to this message
-
Re: expect specific record but get generic
Koert Kuipers 2013-10-21, 21:12
thanks i will look into that
On Mon, Oct 21, 2013 at 4:49 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> On Mon, Oct 21, 2013 at 1:19 PM, Koert Kuipers <[EMAIL PROTECTED]> wrote:
> > doug, could it be a classloader (instead of classpath) issue? looking at
> > spark it seems to run the tasks inside the slaves/workers with a custom
> > classloader.
>
> Yes, it could be a classloader issue.  Perhaps you need to pass the
> Spark task's ClassLoader to SpecificData?  Find where your
> SpecificDatumReader is constructed, and pass it a SpecificData that
> was constructed with that ClassLoader.
>
>
> http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificData.html#SpecificData(java.lang.ClassLoader)
>
> http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificDatumReader.html#SpecificDatumReader(org.apache.avro.specific.SpecificData)
>
> Doug
>
+
Andy He 2014-02-10, 03:58