Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # dev >> IteratorSetting and priorities


Copy link to this message
-
Re: IteratorSetting and priorities
It's because you're building a stack of iterators and the order you set on
the scanner is the order of sources created and passed to init() for each
iterator you create in the stack when the scan is executing on a TServer.
Albeit deprecated, the filtering API in 1.3 does allow you to set multiple
filters at the same priority, though it is broken in certain cases.

The semantics of set up and call order are such that "I want my KVs coming
out of iterator A at priority N to be handled by iterator B at priority N +
1." If you want function composition of your predicates, then increasing
priorities/positions in the stack is the correct approach. I think most, if
not all, of what you want can be accomplished via client side helpers.

For example of where a tree model is used (and you'll be able to see that
tree's can't actually be defined on the client side), check out the
IntersectingIterator.

On Wed, Oct 31, 2012 at 7:52 AM, Patrone, Dennis S. <
[EMAIL PROTECTED]> wrote:

> > The issue with giving multiple iterators the same priority is that the
> API specifies that during the call to init(), one source is given the
> iterator.
>
> I fail to see how this is an issue.  I don't really want a "tree" of
> iterators (I'm not sure how you'd combine the multiple results moving back
> up the tree).  I still want a straight line of iterators, I just don't want
> to have to worry about ordering within a "set" of them at the same priority
> level.
>
> So right now if I add I1 @ priority 1, I2 @ priority 2, and I3 @ priority
> 3, then basically (as I understand it, at least) the output of I1 is fed
> into I2.  Then the output of I2 is fed into I3.
>
> What I want is the API to allow me to add I2 and I3 at priority 2.  Then
> the system has two choices to process my request:
>
> I1 -> I2 -> I3
>
> ...OR...
>
> I1 -> I3 -> I2
>
> Based on my priority values, I don't care which processing chain is
> followed; either is correct.
>
> What I'm NOT asking for is this:
>
>       I2
>     /
> I1
>     \
>       I3
>
> Am I missing something?
>
> Billie- I also looked at ACCUMULO-759.  I need some time later to read
> through it and follow the discussion but then I will try to add something
> coherent.
>
> Thanks,
> Dennis
>
>