Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: QuickSort array out of bound exception during Secondary Sort


Copy link to this message
-
Re: QuickSort array out of bound exception during Secondary Sort
Without access to org.skyz.basic.KeyComparator.compare it is hard to say why this is happening. It looks like you got an array with 0 entries and the given function could not handle that.

I don't really know why.

--Bobby

From: Aseem Anand <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tuesday, February 5, 2013 7:11 AM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: QuickSort array out of bound exception during Secondary Sort

Hi,
I'm facing a problem with hadoop's secondary sort such that it is displaying the following error message. The code I have used has been used by me previously and had not given any issues for a large number of intermediate output pairs. The number of mapper output values is around 175,000. Could someone please guide me on this .

13/02/05 18:34:27 WARN mapred.LocalJobRunner: job_local_0001
java.lang.ArrayIndexOutOfBoundsException: 1
at org.skyz.basic.KeyComparator.compare(KeyComparator.java:23)
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:106)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111)
at org.apache.hadoop.util.QuickSort.fix(QuickSort.java:30)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:83)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:699)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

Hoping for a positive response.
Thanks & Regards,
Aseem Anand
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB