Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Combiner in Secondary Sort


Copy link to this message
-
Re: Combiner in Secondary Sort
Harsh J 2013-03-21, 00:50
You're probably running into
https://issues.apache.org/jira/browse/MAPREDUCE-3310. There was also a
discussion on this IIRC but I cannot find the archive now. One aside
mention on this from Chris is at
http://search-hadoop.com/m/RH5AP11ob2o1.

On Wed, Mar 20, 2013 at 6:10 PM, Fatih Haltas <[EMAIL PROTECTED]> wrote:
> Hi Everyone,
>
> I am trying to implement Secondary Sort Algorithm on mydata. But I am having
> a trouble with my Combiner.
>
> When I donot use Combiner, grouping is done well, I mean one reduce task is
> running for every pair, sharing the same first element.
>
> However, when I set Combiner as Reducer class itself, grouping in the
> Combiner is not done according to my custom GroupingComparator class though.
>
> How can i overrride Combiner class's grouping?
> Or, is there any diagram, showing the sequence of workflow(both in map
> phase, calls of mapper classes and calls of combiner classes and so on
> sequentially) in more detailed?
>
> Thanks very much.

--
Harsh J