Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: QuickSort array out of bound exception during Secondary Sort


Copy link to this message
-
Re: QuickSort array out of bound exception during Secondary Sort
Without access to org.skyz.basic.KeyComparator.compare it is hard to say why this is happening. It looks like you got an array with 0 entries and the given function could not handle that.

I don't really know why.

--Bobby

From: Aseem Anand <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tuesday, February 5, 2013 7:11 AM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: QuickSort array out of bound exception during Secondary Sort

Hi,
I'm facing a problem with hadoop's secondary sort such that it is displaying the following error message. The code I have used has been used by me previously and had not given any issues for a large number of intermediate output pairs. The number of mapper output values is around 175,000. Could someone please guide me on this .

13/02/05 18:34:27 WARN mapred.LocalJobRunner: job_local_0001
java.lang.ArrayIndexOutOfBoundsException: 1
at org.skyz.basic.KeyComparator.compare(KeyComparator.java:23)
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:106)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111)
at org.apache.hadoop.util.QuickSort.fix(QuickSort.java:30)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:83)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:699)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

Hoping for a positive response.
Thanks & Regards,
Aseem Anand