Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> GenericRecord and passivity


Copy link to this message
-
Re: GenericRecord and passivity
Change this line:
DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>(schema_11);

to this:
DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>(schema_10,  schema_11);
On Oct 15, 2013, at 4:01 PM, [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> wrote:

Old Schema: http://pastebin.com/x5CtmKWK
New Schema: http://pastebin.com/c8ZSKMaG
Code to deserialize: http://pastebin.com/RbBWm1VN
On Tue, Oct 15, 2013 at 5:46 PM, Doug Cutting <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
GenericRecord should work well in this context.  Can you provide a
complete example that fails?

Doug

On Tue, Oct 15, 2013 at 3:43 PM, [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
<[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
> Do we know if a GenericRecord is robust to schema evolution? I am currently
> seeing cases where I get an exception like the following if I try to
> deserialize an older record with a newer schema.
>
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 8
>
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
>
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229)
>
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206)
>
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
>
> at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177)
>
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148)
>
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139)
>
>
> The newer schema just has a few more fields added to it and no names from
> the older schema were changed.
>
> Is this a known issue? Should a SpecificRecord be always considered when
> passivity is important?
>
>
> Thanks,
>
>
> --
> Swarnim

--
Swarnim

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB