Hi,

Would someone please explain why ReduceContext.nextKeyValue() creates only a
single instance of the deserializing class?
This is a rather non-standard semantics for deserialization and drove me
insane.

E.g., the following code is rather intuitive but patently wrong; it always
adds a single instance of V to the set.

void reduce(K key, Iterable<V> values, Context context) {
    TreeSet<V> union = new TreeSet<V>();
    for (V v : values) {
        union.add(v);
    }
}

Thanks,

stan

Stan,

Welcome to the reference/reuse hell. Its all part of the learning
process; its documented here and there but everyone learns this by
tearing hair apart I noticed :-)

Have a look at the discussion done previously for this:
http://search-hadoop.com/m/p0VEl1ywwrs1

Also, we had some pointers on how to go about storing values in memory
within mappers/reducers: http://search-hadoop.com/m/ycUTimJIEg and
http://search-hadoop.com/m/uXuH41J1lbA

Hope these help!

On Thu, Aug 11, 2011 at 9:15 AM, Stan Rosenberg
<[EMAIL PROTECTED]> wrote:
> Hi,
>
> Would someone please explain why ReduceContext.nextKeyValue() creates only a
> single instance of the deserializing class?
> This is a rather non-standard semantics for deserialization and drove me
> insane.
>
> E.g., the following code is rather intuitive but patently wrong; it always
> adds a single instance of V to the set.
>
> void reduce(K key, Iterable<V> values, Context context) {
>    TreeSet<V> union = new TreeSet<V>();
>    for (V v : values) {
>        union.add(v);
>    }
> }
>
> Thanks,
>
> stan
>

--
Harsh J

On Thu, Aug 11, 2011 at 1:04 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Stan,
>
> Welcome to the reference/reuse hell. Its all part of the learning
> process; its documented here and there but everyone learns this by
> tearing hair apart I noticed :-)
>
> Have a look at the discussion done previously for this:
> http://search-hadoop.com/m/p0VEl1ywwrs1
>
> Also, we had some pointers on how to go about storing values in memory
> within mappers/reducers: http://search-hadoop.com/m/ycUTimJIEg and
> http://search-hadoop.com/m/uXuH41J1lbA
>
> Hope these help!
 Many thanks!!