Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release


+
Devaraj Das 2013-02-06, 21:53
+
Stack 2013-02-06, 22:29
+
Andrew Purtell 2013-02-06, 23:11
+
Stack 2013-02-06, 23:26
+
Andrew Purtell 2013-02-06, 23:29
+
Nicolas Liochon 2013-02-07, 08:41
+
Stack 2013-02-07, 20:54
+
Jonathan Hsieh 2013-02-06, 22:11
Copy link to this message
-
Re: [DISCUSSION] Sorting out issues for 0.96 for (eventual) release
Stack 2013-02-06, 22:52
On Wed, Feb 6, 2013 at 2:11 PM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:

> Can someone summarize where we are at with the RPC currently?  The KV
> stuff is to make sure we don't regress from core get/put perf
> characteristics of 0.94 right?
>

On RPC/KV Serialiization:

A patch that specifies new pb-based rpc format is up here:
https://issues.apache.org/jira/browse/HBASE-7533  Still TODO is reedit of
rpc spec and implementation of how we will pass blocks of keyvalues.  Its
been spec'd, but as you'd expect, after you've done an implementation, its
funny how the spec needs to change (for example, I thought we could pass
EncodedDataBlocks originally).  I need to finish implementation.

Passing blocks of KVs everywhere will not be done for 0.96.  This would
require reaching up into base Mutation objects client-side and ditto on the
server-side refactoring so throughout the pipeline we do not have
dependencies on the current serialization format; rather we'd rely on the
Cell Interface that is out in hbase-common instead.  This refactoring is a
bunch of work and won't happen anytime soon.  I was just going to do it a
few places to prove the passing of blocks-of-kvs mechanism works.

We have to do blocks of KeyValues rather than passing of pb'd KVs because
pb marshalling/unmarshalling is 10x slower than our current dumb passing of
the kv backing byte array.  We can't keep passing the current kv backing
byte array because KVs need to evolve going forward [1].

St.Ack

1.
https://docs.google.com/document/d/1WEtrq-JTIUhlnlnvA0oYRLp0F8MKpEBeBSCFcQiacdw/edit#heading=h.su4gd8tohaza