Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> ArrayIndexOutOfBoundsException occurs from MapTask

Copy link to this message
RE: ArrayIndexOutOfBoundsException occurs from MapTask
I had also faced same issue, after chnaging "mapreduce.task.io.sort.mb" from 2047 to 512, then jobs are executed success

It might help in your case also..


From: sam liu [[EMAIL PROTECTED]]
Sent: Thursday, September 12, 2013 9:37 AM
Subject: ArrayIndexOutOfBoundsException occurs from MapTask


We are using Hadoop-1.1.1. When run a big workload mapreduce application, it threw a ArrayIndexOutOfBoundsException as below. Seems the issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex, gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and want to know which situations will cause this kind of exception?

Caused by: java.lang.ArrayIndexOutOfBoundsException
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
        at java.io.DataOutputStream.write(DataOutputStream.java:82)
        at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)