Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Reducers without output files


Copy link to this message
-
Re: Reducers without output files
Hi Akro
       You can achieve the same within the existing mapreduce frame work itself. Give a NullWritable in place of reducer output value in reduce function. In your driver class as well mention the output value type as NullWritable.

------Original Message------
From: Arko Provo Mukherjee
To: [EMAIL PROTECTED]
ReplyTo: [EMAIL PROTECTED]
Subject: Reducers without output files
Sent: Sep 15, 2011 08:56

Hello Everyone,

I have a small issue with my Reducer that I am trying to figure out
and wanted some advice.

In the reducer, when writing to the output file as declared in
FileOutputFormat.setOutputPath() I want to write only the key and not
the value when I am calling output.collect().

Is there a way I can ignore the key part?

Else,

Can I write a Reducer function that doesn't do a output.collect()??

Say I omit the FileOutputFormat.setOutputPath() in the Driver Class.

I can then manually write the output to HDFS in the format I like.

Is this legal way to do stuff?

Many thanks in advance!
Warm Regards
Arko
Regards
Bejoy K S
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB