Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> expect specific record but get generic


+
Koert Kuipers 2013-10-21, 15:41
+
Doug Cutting 2013-10-21, 17:07
+
Koert Kuipers 2013-10-21, 20:19
Copy link to this message
-
Re: expect specific record but get generic
On Mon, Oct 21, 2013 at 1:19 PM, Koert Kuipers <[EMAIL PROTECTED]> wrote:
> doug, could it be a classloader (instead of classpath) issue? looking at
> spark it seems to run the tasks inside the slaves/workers with a custom
> classloader.

Yes, it could be a classloader issue.  Perhaps you need to pass the
Spark task's ClassLoader to SpecificData?  Find where your
SpecificDatumReader is constructed, and pass it a SpecificData that
was constructed with that ClassLoader.

http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificData.html#SpecificData(java.lang.ClassLoader)
http://avro.apache.org/docs/current/api/java/org/apache/avro/specific/SpecificDatumReader.html#SpecificDatumReader(org.apache.avro.specific.SpecificData)

Doug
+
Koert Kuipers 2013-10-21, 21:12
+
Andy He 2014-02-10, 03:58
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB