Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: How to output according to the key in reducer?


+
Hemanth Yamijala 2012-09-17, 08:08
Copy link to this message
-
Re: How to output according to the key in reducer?
Hi
Maybe you can refer to
http://hadoop.apache.org/docs/r1.0.3/api/org/apache/hadoop/mapred/lib/MultipleSequenceFileOutputFormat.html

or

http://hadoop.apache.org/docs/r1.0.3/api/org/apache/hadoop/mapred/lib/MultipleTextOutputFormat.html

example like this

  public static class GeneratorOutputFormat extends
      MultipleSequenceFileOutputFormat<IntWritable,Text> {
    // generate a filename based on the segnum stored for this text
    protected String generateFileNameForKeyValue(IntWritable key, Text
value,
        String name) {
      return "ID_" + key.get() + ".dat";
    }

  }

On Mon, Sep 17, 2012 at 3:16 PM, Jason Yang <[EMAIL PROTECTED]>wrote:

> Hi, all
>
> I was wondering how to write all the input with the same key to a single
> file in the reducer ?
>
> say, I got some intermediate outputs from mappers like that:
> key     value
> --------------------
> 1   annie
> 2   Jason
> 1   andy
> 2   Joey
> 1   andrew
> ...
>
> and I would like write all the intermediate outputs with key 1 to file
> "ID_1.dat", and all the intermediate outputs with key 2 to file "ID_2.data".
>
> how could I achieve that?
>
> --
> YANG, Lin
>
>
--
Don't Grow Old, Grow Up... :-)
+
Jason Yang 2012-09-17, 09:23
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB