Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # user - Custom Iterators - behavior when switching tablets


Copy link to this message
-
Custom Iterators - behavior when switching tablets
Slater, David M. 2013-01-22, 16:55
In designing some of my own custom iterators, I was noticing some interesting behavior. Note: my iterator does not return the original key, but instead returns a computed value that is not necessarily in lexicographic order.

So far as I can tell, when the Scanner switches between tablets, it checks the key that is returned in the new tablet and compares it (I think it compares key.row()) with the last key from the previous tablet. If the new key is greater than the previous one, then it proceeds normally. If, however, the new key is less than or equal to the previous key, then the Scanner does not return the value. It does, however, continue to iterate through the tablet, continuing to compare until it finds a key greater than the last one. Once it finds one, however, it progresses through the rest of that tablet without doing a check. (It implicitly assumes that everything in a tablet will be correctly ordered).

Now if I was to return the original key, it would work fine (since it would always be in order), but that also limits the functionality of my custom iterator.

My primary question is: why would it be designed this way? When switching between tablets, are there potential problems that might crop up if this check isn't done?

Thanks,
David
+
Adam Fuchs 2013-01-23, 15:45
+
Mike Drob 2013-01-23, 00:39