Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # user - Custom Iterators - behavior when switching tablets


+
Slater, David M. 2013-01-22, 16:55
+
Adam Fuchs 2013-01-23, 15:45
Copy link to this message
-
Re: Custom Iterators - behavior when switching tablets
Mike Drob 2013-01-23, 00:39
David,

This doesn't answer your design questions, but it might help shed some
light on how to properly handle losing the sort order. Brian did a lot of
work on this in https://issues.apache.org/jira/browse/ACCUMULO-956 so I
highly recommend looking there and comparing to what you've developed.

It's a tricky problem, but I think it is good to get your insights and
experience from it added to the collective knowledge.

Mike
On Tue, Jan 22, 2013 at 11:55 AM, Slater, David M.
<[EMAIL PROTECTED]>wrote:

> In designing some of my own custom iterators, I was noticing some
> interesting behavior. Note: my iterator does not return the original key,
> but instead returns a computed value that is not necessarily in
> lexicographic order.****
>
> ** **
>
> So far as I can tell, when the Scanner switches between tablets, it checks
> the key that is returned in the new tablet and compares it (I think it
> compares key.row()) with the last key from the previous tablet. If the new
> key is greater than the previous one, then it proceeds normally. If,
> however, the new key is less than or equal to the previous key, then the
> Scanner does not return the value. It does, however, continue to iterate
> through the tablet, continuing to compare until it finds a key greater than
> the last one. Once it finds one, however, it progresses through the rest of
> that tablet without doing a check. (It implicitly assumes that everything
> in a tablet will be correctly ordered). ****
>
> ** **
>
> Now if I was to return the original key, it would work fine (since it
> would always be in order), but that also limits the functionality of my
> custom iterator. ****
>
> ** **
>
> My primary question is: why would it be designed this way? When switching
> between tablets, are there potential problems that might crop up if this
> check isn’t done?****
>
> ** **
>
> Thanks,
> David****
>