Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Streaming value of (200MB) from a SequenceFile

Copy link to this message
Streaming value of (200MB) from a SequenceFile
Hi everyone,

I'm having a problem to stream individual key-value pair of 200MB to 1GB
from a MapFile.
I need to stream the large value to an outputstream instead of reading the
entire value before processing because it potentially uses too much memory.

I read the API for MapFile, the next(WritableComparable key, Writable val)
does not return an input stream.

How can I accomplish this?