-MulitpleOutputs outputs just one line
Barak Yaish 2013-01-24, 07:10
I'm trying to utilize MulitpleOutputs ( hadoop 1.0.4 ) to produce multiple
files based on some policy. In the job i set:
MultipleOutputs.addNamedOutput( job, "rejected", TextOutputFormat.class,
Text.class, NullWritable.class );
And at the mapper:
private MultipleOutputs<Text, Writable> mos;
setup(): mos = new MultipleOutputs( context );
map(): if( somecond )
context.write( new Text( key ), NullWritable.get() );
logger.info( "Going to write to mos: " + key );
mos.write( new Text( key ), NullWritable.get(), "/tmp"
The problem I'm facing is that if multiple mappers running that code, I can
see at the logs that the mos.write() is being invoked, but only one line is
printed to the output file under /tmp. Is there some config I missed?