-output files are empty when i turn compression on
Koert Kuipers 2012-04-11, 17:02
i have a simple map-reduce job that i test with only 2 mappers, 2 reducers
and very small input (10 lines of text).
it runs fine without compression. but as soon as i turn on compression
(mapred.compress.map.output=true), the output files (part-00000.snappy,
etc.) are empty. zero records. using logging i can see that my reducer
succesfully calls output.collect(key, value) yet they dont show up in the
file. i tried both snappy and gzip. do i need to do some sort of flushing?
i am on hadoop 0.20.2