Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Streaming value of (200MB) from a SequenceFile


Copy link to this message
-
Streaming value of (200MB) from a SequenceFile
Hi everyone,

I'm having a problem to stream individual key-value pair of 200MB to 1GB
from a MapFile.
I need to stream the large value to an outputstream instead of reading the
entire value before processing because it potentially uses too much memory.

I read the API for MapFile, the next(WritableComparable key, Writable val)
does not return an input stream.

How can I accomplish this?

Thanks,

Jerry
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB