Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - block index comparator


Copy link to this message
-
Re: block index comparator
Matt Corgan 2011-05-27, 20:14
I see now.  Comparator class is written into the HFile.  Got confused
because the tests use a ByteArrayComparator even though most HFiles will use
KeyComparator (i think...)
On Thu, May 26, 2011 at 3:22 PM, Matt Corgan <[EMAIL PROTECTED]> wrote:

> I was looking at some HFile internals and am confused about the block index
> comparator.  It looks like it uses a simple ByteArrayComparator rather than
> a KeyComparator.  Am I missing the comparator being set somewhere?
>
> To test, I added this line to the testBlockContainingKey method of
> TestSeekTo.java which causes the test to fail.
>
> assertEquals(0, reader.blockIndex.blockContainingKey(toKV("cc").getKey(),
> 0, klen));
>
> Even though the key "cc" doesn't exist, shouldn't the block index think
> it's in block 0 because it's before key "i"?  In that test the block index
> thinks it's in block 1 because the first bytes of each key are the
> rowKeyLength, and cc (2) is > i (1)
>
> I must be missing something... maybe actual block indexes use a
> KeyComparator, and that test is just forgetting to set it but lucking out
> because all keys are the same length?
>
> I've been under the assumption that HFile entries are stored on disk in
> KeyComparator order.  Is that correct?
>
> Thanks for any clarifications,
> Matt
>