-Re: Reducers without output files
bejoy.hadoop@... 2011-09-15, 04:09
You can achieve the same within the existing mapreduce frame work itself. Give a NullWritable in place of reducer output value in reduce function. In your driver class as well mention the output value type as NullWritable.
From: Arko Provo Mukherjee
To: [EMAIL PROTECTED]
ReplyTo: [EMAIL PROTECTED]
Subject: Reducers without output files
Sent: Sep 15, 2011 08:56
I have a small issue with my Reducer that I am trying to figure out
and wanted some advice.
In the reducer, when writing to the output file as declared in
FileOutputFormat.setOutputPath() I want to write only the key and not
the value when I am calling output.collect().
Is there a way I can ignore the key part?
Can I write a Reducer function that doesn't do a output.collect()??
Say I omit the FileOutputFormat.setOutputPath() in the Driver Class.
I can then manually write the output to HDFS in the format I like.
Is this legal way to do stuff?
Many thanks in advance!
Bejoy K S