Hi,
    We have been facing an issue when we add new field at the child object
level. In out case we are using the following schema for source.

{
    "namespace":"my.pkg.path.avro",
    "type":"record",
    "name":"PeopleList",
    "fields":[{"name":"Version","type":"int"},
        {
            "name":"People","type":{
                "namespace":"my.pkg.path.avro",
                "type":"array",
"items":{"type":"record","namespace":"my.pkg.path.avro","name":"Person","fields":[
                        {"name":"FirstName","type":"string"},
                        {"name":"LastName","type":"string"}
                    ]
                }
            }
        }
    ]
}

If we add any new field to People record then deserialization failing with
the following error.

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException
        at java.lang.System.arraycopy(Native Method)
        at
org.apache.avro.io.BinaryDecoder.doReadBytes(BinaryDecoder.java:329)
        at
org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:255)
        at
org.apache.avro.io.ValidatingDecoder.readString(ValidatingDecoder.java:107)
        at
org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:344)
        at
org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:337)
        at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:150)
        at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:173)
        at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
        at
org.apache.avro.generic.GenericDatumReader.readArray(GenericDatumReader.java:202)
        at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:146)
        at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:173)
        at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
        at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:135)
        at avro.AvroTest.deserialize(AvroTest.java:140)

I really appreciate any help on this.

Regards,
Krishna

How are you reading the data?  Can you provide a complete example?

To read data you need both the schema that you wrote it with and the
schema you wish to read it as.  DataFileReader and Avro RPC take care
of this for you, but if you're calling the lower-level DatumReader API
directly then you need to be sure to call setSchema with the schema
used to write the data if it differs from that you're using to read
the data.

Doug

On Fri, Nov 9, 2012 at 7:01 AM, krishna prasad <[EMAIL PROTECTED]> wrote:
> Hi,
>     We have been facing an issue when we add new field at the child object
> level. In out case we are using the following schema for source.
>
> {
>     "namespace":"my.pkg.path.avro",
>     "type":"record",
>     "name":"PeopleList",
>     "fields":[{"name":"Version","type":"int"},
>         {
>             "name":"People","type":{
>                 "namespace":"my.pkg.path.avro",
>                 "type":"array",
> "items":{"type":"record","namespace":"my.pkg.path.avro","name":"Person","fields":[
>                         {"name":"FirstName","type":"string"},
>                         {"name":"LastName","type":"string"}
>                     ]
>                 }
>             }
>         }
>     ]
> }
>
> If we add any new field to People record then deserialization failing with
> the following error.
>
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException
>         at java.lang.System.arraycopy(Native Method)
>         at
> org.apache.avro.io.BinaryDecoder.doReadBytes(BinaryDecoder.java:329)
>         at
> org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:255)
>         at
> org.apache.avro.io.ValidatingDecoder.readString(ValidatingDecoder.java:107)
>         at
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:344)
>         at
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:337)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:150)
>         at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:173)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
>         at
> org.apache.avro.generic.GenericDatumReader.readArray(GenericDatumReader.java:202)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:146)
>         at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:173)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
>         at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:135)
>         at avro.AvroTest.deserialize(AvroTest.java:140)
>
> I really appreciate any help on this.
>
> Regards,
> Krishna
>

Hi all,

A java hadoop job that's previously executed without issue began erroring
with the following stack trace - have any of you seen this before?

java.lang.ArrayIndexOutOfBoundsException: 14
        at
org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
        at
org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229)
        at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
        at
org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206)
        at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
        at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:166)
        at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
        at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:129)
        at
org.apache.avro.mapred.AvroSerialization$AvroWrapperDeserializer.deserialize(AvroSerialization.java:83)
        at
org.apache.avro.mapred.AvroSerialization$AvroWrapperDeserializer.deserialize(AvroSerialization.java:65)
        at
org.apache.hadoop.mapred.Task$ValuesIterator.readNextKey(Task.java:1262)
        at
org.apache.hadoop.mapred.Task$ValuesIterator.nextKey(Task.java:1233)
        at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:533)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:429)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)