-Re: MapTask.java -sort part
Harsh J 2013-04-23, 08:49
Yes as Mariappan has already pointed out; the sort is pluggable so the
plug-in can also nullify it. See  for an example implementation of
the MapOutputCollector interface (class MapOutputCopier) to start
with. Per Mariappan, this does not do any sorting, but only merges.
 - http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMerge.java
On Tue, Apr 23, 2013 at 2:00 PM, Samaneh Shokuhi
<[EMAIL PROTECTED]> wrote:
> Hi Harsh,
> That is nice if branch-2 included that .Just one thing i want to be sure
> about that .In this patch MAPREDUCE-4807 (branch-2) , sort in mapper is
> avoided ? Becouse i need a version of hadoop without sorting part in mapper
> and thats the reason i wanted to modify MapTask to disable the mapper sort
> .Do you think branch-2 is appropriate version for me ?
> On Tue, Apr 23, 2013 at 10:01 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>> Note that the Reducer also does a merge sort over the acquired map
>> data; but am not sure if you looked there as well.
>> Mariappan's pointed JIRA (MAPREDUCE-4807) is already available in the
>> 2.0.3+ releases out today and in the current branch-2. It would be
>> simpler to reuse that than do these modifications.
>> On Sun, Apr 21, 2013 at 5:24 PM, Samaneh Shokuhi
>> <[EMAIL PROTECTED]> wrote:
>> > Hi All,
>> > I have modified the MapTask.java file and tried to disable sort part for
>> > some reasons . I have removed this line
>> > *sorter.sort(MapOutputBuffer.this, kvstart, endPosition, reporter); *in
>> > sortAndSpill method but no changes seen in result. I have expected to get
>> > unsorted keys as mapper output but was not like that.
>> > My question is did i perform right action to disable sort or something
>> > needs to be done ?
>> > Samaneh
>> Harsh J