Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Cell Encoders and usage of Cell


+
ramkrishna vasudevan 2013-04-17, 17:16
+
ramkrishna vasudevan 2013-04-18, 07:54
+
Stack 2013-04-19, 01:28
+
ramkrishna vasudevan 2013-04-19, 03:19
Copy link to this message
-
Re: Cell Encoders and usage of Cell
fyi Ram - i started adding the Cell interface to the read path of the delta
encoders in HBASE-7323 <https://issues.apache.org/jira/browse/HBASE-7323>.
 It's one possible place to start working on it.
On Thu, Apr 18, 2013 at 8:19 PM, ramkrishna vasudevan <
[EMAIL PROTECTED]> wrote:

> Thanks for your reply Stack.
> >I think so.  hfile APIs are about KVs.  Should be about Cell I'd think.
> Yes.  This is what i too think.
>
> >If you need the above, you are no doing Cell right I'd argue.  The very
> idea of Cell is a disconnect between how it is stored and Cell use.
>
> Yes Stack.  I understand this.  I am not introducing the getKeyOffset and
> getKeyLength over there.
> My questions were mainly because, if i have the current code  and i would
> want to introduce tags in it, where would i do it?
> So if i need tags to be introduced should i start changing the HFile
> formats also and only then i would be getting the tags to work?
> What do you think here?
>
> > I think the Cell
> Interface needs methods added to allow access to "labels".
> Yes.  You are right.
>
>
>
> On Fri, Apr 19, 2013 at 6:58 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > On Wed, Apr 17, 2013 at 10:16 AM, ramkrishna vasudevan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi
> > >
> > > With the introduction of the new Cell Interface we are providing a way
> > > where both the RPC usage of cell and the usage of Cell in HFile are
> > > unified.(abstracted)
> > >
> > > The current block encoder which encodes the kvs into hfile blocks will
> be
> > > enhanced may be BlockEncode2 which will deal with Cell encoding and the
> > > same will be written to HFile.
> > >
> > >
> > That is the idea.  Current block encoders are unusable for anything but
> > hfile with their presumption of a particular KeyValue serialization and
> >  with hfile context sprinkled throughout.
> >
> >
> >
> > > Does that mean that there are going to be changes to the HFile format
> > also?
> > >  Just to understand is my understanding here correct or not.
> > >
> > >
> > I think so.  hfile APIs are about KVs.  Should be about Cell I'd think.
> >
> >
> >
> > > Because as the Cell interface the row, family, qualifier all are
> treated
> > as
> > > individual byte arrays.  Also it does not provide a way to access the
> > > getKeyOffset() and getKeyLength().
> > >
> > >
> > If you need the above, you are no doing Cell right I'd argue.  The very
> > idea of Cell is a disconnect between how it is stored and Cell use.
> >
> >
> >
> >
> > > This is in lieu with HBASE-7448 - Adding tags to cell interface and
> then
> > > the same will  be used in
> > > HBASE-7663 - Visibility labels.
> > >
> > >
> > I am not sure I follow what you are asking above Ram.  I think the Cell
> > Interface needs methods added to allow access to "labels".
> >
> > St.Ack
> >
> >
> > > May be further queries/doubts can be posted on those relevant JIRAs to
> > > proceed work on that.
> > >
> > > Regards
> > > Ram
> > >
> >
>
+
Nick Dimiduk 2013-04-22, 00:08
+
Matt Corgan 2013-04-22, 00:36
+
ramkrishna vasudevan 2013-04-22, 03:58
+
Stack 2013-04-22, 19:28
+
Andrew Purtell 2013-04-22, 22:16
+
ramkrishna vasudevan 2013-04-23, 06:40