Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> expect specific record but get generic


Copy link to this message
-
Re: expect specific record but get generic
On Mon, Oct 21, 2013 at 1:19 PM, Koert Kuipers <[EMAIL PROTECTED]> wrote:
> doug, could it be a classloader (instead of classpath) issue? looking at
> spark it seems to run the tasks inside the slaves/workers with a custom
> classloader.

Yes, it could be a classloader issue.  Perhaps you need to pass the
Spark task's ClassLoader to SpecificData?  Find where your
SpecificDatumReader is constructed, and pass it a SpecificData that
was constructed with that ClassLoader.

http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificData.html#SpecificData(java.lang.ClassLoader)
http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificDatumReader.html#SpecificDatumReader(org.apache.avro.specific.SpecificData)

Doug
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB