Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> ArrayIndexOutOfBoundsException in Symbol.getSymbol in map reduce job


+
Sripad Sriram 2013-05-13, 21:58
+
Harsh J 2013-05-13, 22:40
Copy link to this message
-
Re: ArrayIndexOutOfBoundsException in Symbol.getSymbol in map reduce job
I'm using Avro 1.7.4, and the error is coming between a map step and reduce
step. I'm unsure how an Avro record could get corrupted between a mapper
and reducer, but it has, how would I be able to handle that exception?
On Mon, May 13, 2013 at 3:40 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> Its difficult to tell what the error means without context and other
> info (such as version). If I had to guess, I think there may be a
> corruption on the file being processed here. Does running the file
> through avro-tools' tojson sub-command end up in a successful read?
>
> On Tue, May 14, 2013 at 3:28 AM, Sripad Sriram <[EMAIL PROTECTED]> wrote:
> > Hi all,
> >
> > A java hadoop job that's previously executed without issue began erroring
> > with the following stack trace - have any of you seen this before?
> >
> > java.lang.ArrayIndexOutOfBoundsException: 14
> >         at
> > org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> >         at
> > org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:229)
> >         at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> >         at
> > org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:206)
> >         at
> >
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
> >         at
> >
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:166)
> >         at
> >
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:138)
> >         at
> >
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:129)
> >         at
> >
> org.apache.avro.mapred.AvroSerialization$AvroWrapperDeserializer.deserialize(AvroSerialization.java:83)
> >         at
> >
> org.apache.avro.mapred.AvroSerialization$AvroWrapperDeserializer.deserialize(AvroSerialization.java:65)
> >         at
> > org.apache.hadoop.mapred.Task$ValuesIterator.readNextKey(Task.java:1262)
> >         at
> > org.apache.hadoop.mapred.Task$ValuesIterator.nextKey(Task.java:1233)
> >         at
> > org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:533)
> >         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:429)
> >         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at javax.security.auth.Subject.doAs(Subject.java:396)
> >         at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
> >         at org.apache.hadoop.mapred.Child.main(Child.java:249)
>
>
>
> --
> Harsh J
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB