Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Reducers without output files


Copy link to this message
-
Re: Reducers without output files
bejoy.hadoop@... 2011-09-15, 04:09
Hi Akro
       You can achieve the same within the existing mapreduce frame work itself. Give a NullWritable in place of reducer output value in reduce function. In your driver class as well mention the output value type as NullWritable.

------Original Message------
From: Arko Provo Mukherjee
To: [EMAIL PROTECTED]
ReplyTo: [EMAIL PROTECTED]
Subject: Reducers without output files
Sent: Sep 15, 2011 08:56

Hello Everyone,

I have a small issue with my Reducer that I am trying to figure out
and wanted some advice.

In the reducer, when writing to the output file as declared in
FileOutputFormat.setOutputPath() I want to write only the key and not
the value when I am calling output.collect().

Is there a way I can ignore the key part?

Else,

Can I write a Reducer function that doesn't do a output.collect()??

Say I omit the FileOutputFormat.setOutputPath() in the Driver Class.

I can then manually write the output to HDFS in the format I like.

Is this legal way to do stuff?

Many thanks in advance!
Warm Regards
Arko
Regards
Bejoy K S