Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - ArrayIndexOutOfBoundsException when adding new field at child object level in the source


+
krishna prasad 2012-11-09, 15:01
Copy link to this message
-
Re: ArrayIndexOutOfBoundsException when adding new field at child object level in the source
Doug Cutting 2012-11-09, 17:54
How are you reading the data?  Can you provide a complete example?

To read data you need both the schema that you wrote it with and the
schema you wish to read it as.  DataFileReader and Avro RPC take care
of this for you, but if you're calling the lower-level DatumReader API
directly then you need to be sure to call setSchema with the schema
used to write the data if it differs from that you're using to read
the data.

Doug

On Fri, Nov 9, 2012 at 7:01 AM, krishna prasad <[EMAIL PROTECTED]> wrote:
> Hi,
>     We have been facing an issue when we add new field at the child object
> level. In out case we are using the following schema for source.
>
> {
>     "namespace":"my.pkg.path.avro",
>     "type":"record",
>     "name":"PeopleList",
>     "fields":[{"name":"Version","type":"int"},
>         {
>             "name":"People","type":{
>                 "namespace":"my.pkg.path.avro",
>                 "type":"array",
> "items":{"type":"record","namespace":"my.pkg.path.avro","name":"Person","fields":[
>                         {"name":"FirstName","type":"string"},
>                         {"name":"LastName","type":"string"}
>                     ]
>                 }
>             }
>         }
>     ]
> }
>
> If we add any new field to People record then deserialization failing with
> the following error.
>
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException
>         at java.lang.System.arraycopy(Native Method)
>         at
> org.apache.avro.io.BinaryDecoder.doReadBytes(BinaryDecoder.java:329)
>         at
> org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:255)
>         at
> org.apache.avro.io.ValidatingDecoder.readString(ValidatingDecoder.java:107)
>         at
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:344)
>         at
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:337)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:150)
>         at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:173)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
>         at
> org.apache.avro.generic.GenericDatumReader.readArray(GenericDatumReader.java:202)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:146)
>         at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:173)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:135)
>         at avro.AvroTest.deserialize(AvroTest.java:140)
>
> I really appreciate any help on this.
>
> Regards,
> Krishna
>
+
Sripad Sriram 2013-05-13, 21:58