Combiners seem to be a big improvement on the method of implementing what were once termed aggregating iterators. What's the current (1.4 on) best way to implement a filtering iterator?

Aaron

On Thursday, February 9, 2012 12:26:01 PM, "Aaron Cordova" <[EMAIL PROTECTED]> wrote:
> Combiners seem to be a big improvement on the method of implementing
> what were once termed aggregating iterators. What's the current (1.4
> on) best way to implement a filtering iterator?

Prior to 1.4, there was a FilteringIterator into which one would plug filters implementing the org.apache.accumulo.core.iterators.filter.Filter interface.
This was confusing from a configuration perspective because the filters were not actually Iterators, but were options on the FilteringIterator.

In 1.4, we have introduced org.apache.accumulo.core.iterators.Filter (note the package difference) which is an Iterator.  Now filters extend this abstract Filter class.  The relevant method one must implement is the same, i.e.
public boolean accept(Key k, Value v), so converting filters of the old type to the new type will be relatively easy.  Converting aggregators to combiners requires a bit more thought.

We have also made progress in formalizing the desired behavior of Iterator methods and usage patterns.  See the new javadocs for SortedKeyValueIterator.  These were added in 1.3, but 1.4 begins to enforce them for Iterators that extend the WrappingIterator.  An example is that data should not be read from the the source iterator in the init method; it can be assumed that seek will always be called before data is read from an iterator.  Also, all iterators that Accumulo provides for users out of the box are now located in the org.apache.accumulo.core.iterators.user package.

Billie

Thanks for the answer.

What would one use if one wanted to apply some arbitrary transformation to all the values? Say, converting them to ints, calculate their square, and convert back to bytes?