Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Can caller mutate value after calling OutputCollector.collect(key,value) ?


Copy link to this message
-
Re: Can caller mutate value after calling OutputCollector.collect(key,value) ?
Hi

On Sun, Feb 12, 2012 at 8:58 PM, Mike Spreitzer <[EMAIL PROTECTED]> wrote:
> I have a question about the contract in the
> org.apache.hadoop.mapred.OutputCollector interface.  If it matters, let us
> say we are talking about Hadoop-1.0.0.  In my map or reduce method, after it
> calls output.collect(key,value), is it allowed to mutate (side effect)
> either the key or the value?
>
> If the answer is "it depends on the OutputFormat", then what is the answer
> for the more prominent ones, such as SequenceFileFormat and
> TextOutputFormat?

In both cases the key is serialized immediately so the caller cannot
mutate the key after calling collect.

http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0/src/mapred/org/apache/hadoop/mapred/TextOutputFormat.java
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0/src/mapred/org/apache/hadoop/mapred/SequenceFileOutputFormat.java
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0/src/core/org/apache/hadoop/io/SequenceFile.java

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/