Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> block index comparator


Copy link to this message
-
Re: block index comparator
I see now.  Comparator class is written into the HFile.  Got confused
because the tests use a ByteArrayComparator even though most HFiles will use
KeyComparator (i think...)
On Thu, May 26, 2011 at 3:22 PM, Matt Corgan <[EMAIL PROTECTED]> wrote:

> I was looking at some HFile internals and am confused about the block index
> comparator.  It looks like it uses a simple ByteArrayComparator rather than
> a KeyComparator.  Am I missing the comparator being set somewhere?
>
> To test, I added this line to the testBlockContainingKey method of
> TestSeekTo.java which causes the test to fail.
>
> assertEquals(0, reader.blockIndex.blockContainingKey(toKV("cc").getKey(),
> 0, klen));
>
> Even though the key "cc" doesn't exist, shouldn't the block index think
> it's in block 0 because it's before key "i"?  In that test the block index
> thinks it's in block 1 because the first bytes of each key are the
> rowKeyLength, and cc (2) is > i (1)
>
> I must be missing something... maybe actual block indexes use a
> KeyComparator, and that test is just forgetting to set it but lucking out
> because all keys are the same length?
>
> I've been under the assumption that HFile entries are stored on disk in
> KeyComparator order.  Is that correct?
>
> Thanks for any clarifications,
> Matt
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB