Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Problem with HFile lexical comparison


Copy link to this message
-
Re: Problem with HFile lexical comparison
Rohit Kelkar 2013-06-20, 00:11
Thanks for the replies, I tried the KeyValue.KVComparator but still no
luck. So I commented the comparator and played around with the sequence of
writing the qualifiers to the HFile. (see code here:
https://gist.github.com/anonymous/5819254)

If I set the variable String[] starr = new String[]{"a", "d", "dt", "dth"}
then the code breaks while writing the qualifier "dt" to the HFile.
If I set the variable String[] starr = new String[]{"a", "dth", "dt", "d"}
then the code runs successfully.
If I set the variable String[] starr = new String[]{"dth", "dt", "d", "a"}
then the code breaks while writing "a" to the HFile

Does this mean that if the qualifiers start with the same character then
the longest qualifier should be written first? Else the usual lexical order
is honoured?

The code throws following stack trace
Added a key not lexically larger than previous
key=\x00\x1B10011-2-0000000000000000703\x02sddt\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x04,
lastkey=\x00\x1B10011-2-0000000000000000703\x02sdd\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x04
 at
org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:207)
at
org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:317)
 at
org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:282)
at com.mycompany.hbase.process.myprocess.myFunction(MyClass.java:1492)

I am using hbase-0.94.2

- Rohit Kelkar
On Wed, Jun 19, 2013 at 1:15 PM, Jeff Kolesky <[EMAIL PROTECTED]> wrote:

> I believe you need to use the KVComparator:
>
>
> https://github.com/apache/hbase/blob/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L88
>
> Jeff
>
>
> On Wed, Jun 19, 2013 at 10:32 AM, Rohit Kelkar <[EMAIL PROTECTED]
> >wrote:
>
> > Here is the code - https://gist.github.com/anonymous/5816180
> >
> > I guess the issue is with my use of the comparator function.
> >
> > - R
> >
>
>
>
> --
> *Jeff Kolesky*
> Chief Software Architect
> *Opower*
>