Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> ArrayIndexOutOfBoundsException in Symbol.getSymbol in map reduce job


+
Sripad Sriram 2013-05-13, 21:58
+
Harsh J 2013-05-13, 22:40
Copy link to this message
-
Re: ArrayIndexOutOfBoundsException in Symbol.getSymbol in map reduce job
I'm using Avro 1.7.4, and the error is coming between a map step and reduce
step. I'm unsure how an Avro record could get corrupted between a mapper
and reducer, but it has, how would I be able to handle that exception?
On Mon, May 13, 2013 at 3:40 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Its difficult to tell what the error means without context and other
> info (such as version). If I had to guess, I think there may be a
> corruption on the file being processed here. Does running the file
> through avro-tools' tojson sub-command end up in a successful read?
>
> On Tue, May 14, 2013 at 3:28 AM, Sripad Sriram <[EMAIL PROTECTED]> wrote:
> > Hi all,
> >
> > A java hadoop job that's previously executed without issue began erroring
> > with the following stack trace - have any of you seen this before?
> >
> > java.lang.ArrayIndexOutOfBoundsException: 14
> >         at
> > org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> >         at
> > org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229)
> >         at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> >         at
> > org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206)
> >         at
> >
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
> >         at
> >
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:166)
> >         at
> >
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
> >         at
> >
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:129)
> >         at
> >
> org.apache.avro.mapred.AvroSerialization$AvroWrapperDeserializer.deserialize(AvroSerialization.java:83)
> >         at
> >
> org.apache.avro.mapred.AvroSerialization$AvroWrapperDeserializer.deserialize(AvroSerialization.java:65)
> >         at
> > org.apache.hadoop.mapred.Task$ValuesIterator.readNextKey(Task.java:1262)
> >         at
> > org.apache.hadoop.mapred.Task$ValuesIterator.nextKey(Task.java:1233)
> >         at
> > org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:533)
> >         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:429)
> >         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at javax.security.auth.Subject.doAs(Subject.java:396)
> >         at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
> >         at org.apache.hadoop.mapred.Child.main(Child.java:249)
>
>
>
> --
> Harsh J
>