Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - org.apache.accumulo.core.iterators.Combiner: key scope?


Copy link to this message
-
Re: org.apache.accumulo.core.iterators.Combiner: key scope?
Aaron Cordova 2012-03-20, 12:46
Returning 0 to 1 KV pair or just value would be nice, and less of a change than 0 to N KV pairs

On Mar 19, 2012, at 4:35 PM, Keith Turner wrote:

> On Mon, Mar 19, 2012 at 4:09 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
>> The original semantics of reduce() from lisp is to compact everything down into one object .. but the original MapReduce semantics allow reduce and map functions to emit() as many new KV pairs as one desires. To bring Accumulo's reduce() function closer to the usage of MapReduce's reduce() might not introduce a huge amount of cognitive load on users, especially if they are coming from the MapReduce world.
>
> Another thing that map reduce allows is for a reducer to emit zero KV.
> Users have asked if this was possible in a combiner/aggregator
> before, the ability to filter.  Allowing a combiner to do this can be
> more efficient than a Combiner+Filter, because the Filter may need to
> redo computation that the Combiner just did inorder to make a
> decision.
>
> Keith