Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Partitioning Reducer Output


Copy link to this message
-
Re: Partitioning Reducer Output
On 04/02/2010 08:32 PM, rakesh kothari wrote:
>
> Hi,
>
> What's the best way to partition data generated from Reducer into multiple > directories in Hadoop 0.20.1. I was thinking of using MultipleTextOutputFor> mat but that's not backward compatible with other API's in this version of > hadoop.
>
> Thanks,
> -Rakesh  

Use a partitioner?

http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/mapreduce/Job.html#setPartitionerClass%28java.lang.Class%29

HTH,

DR
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB