Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> Custom Iterators - behavior when switching tablets


Copy link to this message
-
Re: Custom Iterators - behavior when switching tablets
David,

This doesn't answer your design questions, but it might help shed some
light on how to properly handle losing the sort order. Brian did a lot of
work on this in https://issues.apache.org/jira/browse/ACCUMULO-956 so I
highly recommend looking there and comparing to what you've developed.

It's a tricky problem, but I think it is good to get your insights and
experience from it added to the collective knowledge.

Mike
On Tue, Jan 22, 2013 at 11:55 AM, Slater, David M.
<[EMAIL PROTECTED]>wrote:

> In designing some of my own custom iterators, I was noticing some
> interesting behavior. Note: my iterator does not return the original key,
> but instead returns a computed value that is not necessarily in
> lexicographic order.****
>
> ** **
>
> So far as I can tell, when the Scanner switches between tablets, it checks
> the key that is returned in the new tablet and compares it (I think it
> compares key.row()) with the last key from the previous tablet. If the new
> key is greater than the previous one, then it proceeds normally. If,
> however, the new key is less than or equal to the previous key, then the
> Scanner does not return the value. It does, however, continue to iterate
> through the tablet, continuing to compare until it finds a key greater than
> the last one. Once it finds one, however, it progresses through the rest of
> that tablet without doing a check. (It implicitly assumes that everything
> in a tablet will be correctly ordered). ****
>
> ** **
>
> Now if I was to return the original key, it would work fine (since it
> would always be in order), but that also limits the functionality of my
> custom iterator. ****
>
> ** **
>
> My primary question is: why would it be designed this way? When switching
> between tablets, are there potential problems that might crop up if this
> check isn’t done?****
>
> ** **
>
> Thanks,
> David****
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB