Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Cell Encoders and usage of Cell


+
ramkrishna vasudevan 2013-04-17, 17:16
+
ramkrishna vasudevan 2013-04-18, 07:54
+
Stack 2013-04-19, 01:28
+
ramkrishna vasudevan 2013-04-19, 03:19
Copy link to this message
-
Re: Cell Encoders and usage of Cell
fyi Ram - i started adding the Cell interface to the read path of the delta
encoders in HBASE-7323 <https://issues.apache.org/jira/browse/HBASE-7323>.
 It's one possible place to start working on it.
On Thu, Apr 18, 2013 at 8:19 PM, ramkrishna vasudevan <
[EMAIL PROTECTED]> wrote:

> Thanks for your reply Stack.
> >I think so.  hfile APIs are about KVs.  Should be about Cell I'd think.
> Yes.  This is what i too think.
>
> >If you need the above, you are no doing Cell right I'd argue.  The very
> idea of Cell is a disconnect between how it is stored and Cell use.
>
> Yes Stack.  I understand this.  I am not introducing the getKeyOffset and
> getKeyLength over there.
> My questions were mainly because, if i have the current code  and i would
> want to introduce tags in it, where would i do it?
> So if i need tags to be introduced should i start changing the HFile
> formats also and only then i would be getting the tags to work?
> What do you think here?
>
> > I think the Cell
> Interface needs methods added to allow access to "labels".
> Yes.  You are right.
>
>
>
> On Fri, Apr 19, 2013 at 6:58 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > On Wed, Apr 17, 2013 at 10:16 AM, ramkrishna vasudevan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi
> > >
> > > With the introduction of the new Cell Interface we are providing a way
> > > where both the RPC usage of cell and the usage of Cell in HFile are
> > > unified.(abstracted)
> > >
> > > The current block encoder which encodes the kvs into hfile blocks will
> be
> > > enhanced may be BlockEncode2 which will deal with Cell encoding and the
> > > same will be written to HFile.
> > >
> > >
> > That is the idea.  Current block encoders are unusable for anything but
> > hfile with their presumption of a particular KeyValue serialization and
> >  with hfile context sprinkled throughout.
> >
> >
> >
> > > Does that mean that there are going to be changes to the HFile format
> > also?
> > >  Just to understand is my understanding here correct or not.
> > >
> > >
> > I think so.  hfile APIs are about KVs.  Should be about Cell I'd think.
> >
> >
> >
> > > Because as the Cell interface the row, family, qualifier all are
> treated
> > as
> > > individual byte arrays.  Also it does not provide a way to access the
> > > getKeyOffset() and getKeyLength().
> > >
> > >
> > If you need the above, you are no doing Cell right I'd argue.  The very
> > idea of Cell is a disconnect between how it is stored and Cell use.
> >
> >
> >
> >
> > > This is in lieu with HBASE-7448 - Adding tags to cell interface and
> then
> > > the same will  be used in
> > > HBASE-7663 - Visibility labels.
> > >
> > >
> > I am not sure I follow what you are asking above Ram.  I think the Cell
> > Interface needs methods added to allow access to "labels".
> >
> > St.Ack
> >
> >
> > > May be further queries/doubts can be posted on those relevant JIRAs to
> > > proceed work on that.
> > >
> > > Regards
> > > Ram
> > >
> >
>
+
Nick Dimiduk 2013-04-22, 00:08
+
Matt Corgan 2013-04-22, 00:36
+
ramkrishna vasudevan 2013-04-22, 03:58
+
Stack 2013-04-22, 19:28
+
Andrew Purtell 2013-04-22, 22:16
+
ramkrishna vasudevan 2013-04-23, 06:40
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB