Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Reduce Task Clarification


Copy link to this message
-
Re: Reduce Task Clarification
Implement raw comparator for your emitted keys to sort the output at the
reducer.

::::::::::::::::::::::::::::::::::::::::
Raj K Singh
http://www.rajkrrsingh.blogspot.com
Mobile  Tel: +91 (0)9899821370
On Wed, Aug 14, 2013 at 1:21 AM, Sam Garrett <[EMAIL PROTECTED]> wrote:

> I am working on a MapReduce job where I would like to have the output
> sorted by a LongWritable value. I read the Anatomy of a MapReduce Run in
> the Definitive Guide and it didn't say explicitly whether reduce() gets
> called only once per map output key. If it does get called only once I was
> thinking that I could use this:
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapreduce/Job.html#setSortComparatorClass(java.lang.Class)to do the sorting.
>
> Thank you for your time.
>
> --
> Sam Garrett
> ActionX, NYC
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB