Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - GenericRecord and passivity


Copy link to this message
-
Re: GenericRecord and passivity
Eric Wasserman 2013-10-15, 23:49
Change this line:
DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>(schema_11);

to this:
DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>(schema_10,  schema_11);
On Oct 15, 2013, at 4:01 PM, [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]> wrote:

Old Schema: http://pastebin.com/x5CtmKWK
New Schema: http://pastebin.com/c8ZSKMaG
Code to deserialize: http://pastebin.com/RbBWm1VN
On Tue, Oct 15, 2013 at 5:46 PM, Doug Cutting <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
GenericRecord should work well in this context.  Can you provide a
complete example that fails?

Doug

On Tue, Oct 15, 2013 at 3:43 PM, [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
<[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
> Do we know if a GenericRecord is robust to schema evolution? I am currently
> seeing cases where I get an exception like the following if I try to
> deserialize an older record with a newer schema.
>
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 8
>
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
>
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229)
>
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206)
>
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
>
> at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177)
>
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148)
>
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139)
>
>
> The newer schema just has a few more fields added to it and no names from
> the older schema were changed.
>
> Is this a known issue? Should a SpecificRecord be always considered when
> passivity is important?
>
>
> Thanks,
>
>
> --
> Swarnim

--
Swarnim