Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Problem with HFile lexical comparison


Copy link to this message
-
Re: Problem with HFile lexical comparison
Perfect. That worked. Thanks.

- R
On Wed, Jun 19, 2013 at 7:23 PM, Jeff Kolesky <[EMAIL PROTECTED]> wrote:

> Last time I wrote directly to an HFile, I instantiated an HFile.Writer
> using this statement:
>
>         HFile.Writer writer = HFile.getWriterFactory(config)
>             .createWriter(fs, hfilePath,
>                     (bytesPerBlock * 1024),
>                     Compression.Algorithm.GZ,
>                     KeyValue.KEY_COMPARATOR);
>
> Perhaps you need the declaration of the comparator in the create statement
> for the writer.
>
> Jeff
>
>
>
> On Wed, Jun 19, 2013 at 5:11 PM, Rohit Kelkar <[EMAIL PROTECTED]>
> wrote:
>
> > Thanks for the replies, I tried the KeyValue.KVComparator but still no
> > luck. So I commented the comparator and played around with the sequence
> of
> > writing the qualifiers to the HFile. (see code here:
> > https://gist.github.com/anonymous/5819254)
> >
> > If I set the variable String[] starr = new String[]{"a", "d", "dt",
> "dth"}
> > then the code breaks while writing the qualifier "dt" to the HFile.
> > If I set the variable String[] starr = new String[]{"a", "dth", "dt",
> "d"}
> > then the code runs successfully.
> > If I set the variable String[] starr = new String[]{"dth", "dt", "d",
> "a"}
> > then the code breaks while writing "a" to the HFile
> >
> > Does this mean that if the qualifiers start with the same character then
> > the longest qualifier should be written first? Else the usual lexical
> order
> > is honoured?
> >
> > The code throws following stack trace
> > Added a key not lexically larger than previous
> >
> >
> key=\x00\x1B10011-2-0000000000000000703\x02sddt\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x04,
> >
> >
> lastkey=\x00\x1B10011-2-0000000000000000703\x02sdd\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x04
> >  at
> >
> >
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:207)
> > at
> >
> >
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:317)
> >  at
> >
> >
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:282)
> > at com.mycompany.hbase.process.myprocess.myFunction(MyClass.java:1492)
> >
> > I am using hbase-0.94.2
> >
> > - Rohit Kelkar
> >
> >
> > On Wed, Jun 19, 2013 at 1:15 PM, Jeff Kolesky <[EMAIL PROTECTED]> wrote:
> >
> > > I believe you need to use the KVComparator:
> > >
> > >
> > >
> >
> https://github.com/apache/hbase/blob/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L88
> > >
> > > Jeff
> > >
> > >
> > > On Wed, Jun 19, 2013 at 10:32 AM, Rohit Kelkar <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Here is the code - https://gist.github.com/anonymous/5816180
> > > >
> > > > I guess the issue is with my use of the comparator function.
> > > >
> > > > - R
> > > >
> > >
> > >
> > >
> > > --
> > > *Jeff Kolesky*
> > > Chief Software Architect
> > > *Opower*
> > >
> >
>
>
>
> --
> *Jeff Kolesky*
> Chief Software Architect
> *Opower*
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB