Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - ArrayIndexOutOfBoundsException occurs from MapTask


Copy link to this message
-
RE: ArrayIndexOutOfBoundsException occurs from MapTask
Brahma Reddy Battula 2013-09-12, 01:59
I had also faced same issue, after chnaging "mapreduce.task.io.sort.mb" from 2047 to 512, then jobs are executed success

It might help in your case also..

________________________________

From: sam liu [[EMAIL PROTECTED]]
Sent: Thursday, September 12, 2013 9:37 AM
To: [EMAIL PROTECTED]
Subject: ArrayIndexOutOfBoundsException occurs from MapTask

Hi,

We are using Hadoop-1.1.1. When run a big workload mapreduce application, it threw a ArrayIndexOutOfBoundsException as below. Seems the issue was threw in line 'System.arraycopy(b, off, kvbuffer, bufindex, gaplen);' in MapTask. I am not familiar with the code of MapTask.java, and want to know which situations will cause this kind of exception?

Caused by: java.lang.ArrayIndexOutOfBoundsException
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1270)
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1193)
        at java.io.DataOutputStream.write(DataOutputStream.java:82)
        at java.io.DataOutputStream.writeInt(DataOutputStream.java:191)

Thanks!