Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Can caller mutate value after calling OutputCollector.collect(key,value) ?


Copy link to this message
-
Re: Can caller mutate value after calling OutputCollector.collect(key,value) ?
Hi

On Sun, Feb 12, 2012 at 8:58 PM, Mike Spreitzer <[EMAIL PROTECTED]> wrote:
> I have a question about the contract in the
> org.apache.hadoop.mapred.OutputCollector interface.  If it matters, let us
> say we are talking about Hadoop-1.0.0.  In my map or reduce method, after it
> calls output.collect(key,value), is it allowed to mutate (side effect)
> either the key or the value?
>
> If the answer is "it depends on the OutputFormat", then what is the answer
> for the more prominent ones, such as SequenceFileFormat and
> TextOutputFormat?

In both cases the key is serialized immediately so the caller cannot
mutate the key after calling collect.

http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0/src/mapred/org/apache/hadoop/mapred/TextOutputFormat.java
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0/src/mapred/org/apache/hadoop/mapred/SequenceFileOutputFormat.java
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0/src/core/org/apache/hadoop/io/SequenceFile.java

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB