Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Combiner in Secondary Sort


Copy link to this message
-
Re: Combiner in Secondary Sort
You're probably running into
https://issues.apache.org/jira/browse/MAPREDUCE-3310. There was also a
discussion on this IIRC but I cannot find the archive now. One aside
mention on this from Chris is at
http://search-hadoop.com/m/RH5AP11ob2o1.

On Wed, Mar 20, 2013 at 6:10 PM, Fatih Haltas <[EMAIL PROTECTED]> wrote:
> Hi Everyone,
>
> I am trying to implement Secondary Sort Algorithm on mydata. But I am having
> a trouble with my Combiner.
>
> When I donot use Combiner, grouping is done well, I mean one reduce task is
> running for every pair, sharing the same first element.
>
> However, when I set Combiner as Reducer class itself, grouping in the
> Combiner is not done according to my custom GroupingComparator class though.
>
> How can i overrride Combiner class's grouping?
> Or, is there any diagram, showing the sequence of workflow(both in map
> phase, calls of mapper classes and calls of combiner classes and so on
> sequentially) in more detailed?
>
> Thanks very much.

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB