Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> HBase wire compatibility


+
Jimmy Xiang 2012-02-13, 19:01
+
Ted Yu 2012-02-13, 19:03
+
Jimmy Xiang 2012-02-13, 21:02
+
Ted Yu 2012-02-13, 21:18
+
Ted Yu 2012-02-13, 21:23
+
Jimmy Xiang 2012-02-13, 22:04
+
Ted Yu 2012-02-13, 22:07
+
Todd Lipcon 2012-02-13, 22:22
+
Ted Yu 2012-02-13, 22:33
+
Andrew Purtell 2012-02-13, 23:23
+
Stack 2012-02-14, 00:09
+
Ted Yu 2012-02-14, 00:15
+
Jimmy Xiang 2012-02-14, 00:58
+
Stack 2012-02-14, 01:10
+
Jimmy Xiang 2012-02-14, 01:21
+
Gregory Chanan 2012-02-15, 01:30
+
Todd Lipcon 2012-02-15, 01:57
+
Ted Yu 2012-02-15, 20:09
+
lars hofhansl 2012-02-15, 20:12
+
Ted Yu 2012-02-15, 22:01
+
Stack 2012-02-15, 03:36
+
Stack 2012-02-21, 04:53
Copy link to this message
-
Re: HBase wire compatibility
Good writeup, Jimmy (was away for a few days due to an event in my family)
 
Some quick questions - Has there been any thoughts on the plan to use HBASE-5394. Are we going to make the hbase protocols (like HRegionInterface) protobuf aware?

In Hadoop, I have seen the following:
1. In HDFS, the protocol definitions are not changed (like org.apache.hadoop.hdfs.protocol.ClientProtocol). Instead there are translators that are defined that implement the mapping of protobuf datastructures to application-level datastructures and vice versa (for example, have a look at ClientNamenodeProtocolTranslatorPB and ClientNamenodeProtocolServerSideTranslatorPB in the package org.apache.hadoop.hdfs.protocolPB).
2. In Yarn (MRV2), all protocol definitions are written in PB
 
Since the base RPC still uses writables for payload encoding, a translation happens when the protobuf objects are sent/received (as an example look at org.apache.hadoop.ipc.ProtobufRpcEngine; classes RpcRequestWritable and RpcResponseWritable).
 
What does the HBase community think about the above?
On Feb 13, 2012, at 1:02 PM, Jimmy Xiang wrote:

> I posted the proposal on wiki:
>
> http://wiki.apache.org/hadoop/Hbase/HBaseWireCompatibility
>
> Thanks,
> Jimmy
>
> On Mon, Feb 13, 2012 at 11:03 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Can you post on wiki ?
>>
>> Attachment stripped.
>>
>> On Mon, Feb 13, 2012 at 11:01 AM, Jimmy Xiang <[EMAIL PROTECTED]> wrote:
>>
>>> Hello,
>>>
>>> As HBase installation base is getting bigger, we are ready to work on the
>>> wire compatibility issue.
>>> The goal is to make HBase easier for operators to upgrade, while it is
>>> also easier for developers to
>>> enhance, re-architect if necessary.
>>>
>>> The attached is a proposal we came up.  We'd like to start with two
>> phases:
>>>
>>> Phase 1: Compatibility between client applications and HBase clusters
>>> Phase 2: HBase cluster rolling upgrade within same major version
>>>
>>> Could you please review?
>>>
>>> Thanks,
>>> Jimmy
>>>
>>
+
Todd Lipcon 2012-02-16, 20:48
+
Jimmy Xiang 2012-02-16, 20:56
+
Jacques 2012-02-16, 22:27
+
Todd Lipcon 2012-02-16, 22:33
+
Jacques 2012-02-16, 22:41
+
Gregory Chanan 2012-02-16, 22:58
+
Jeff Whiting 2012-02-16, 23:04
+
Todd Lipcon 2012-02-16, 23:09
+
Dhruba Borthakur 2012-02-16, 23:11
+
Jeff Whiting 2012-02-16, 23:55
+
Stack 2012-02-17, 00:54
+
Enis Söztutar 2012-02-18, 01:30
+
Gregory Chanan 2012-02-18, 01:42
+
Enis Söztutar 2012-02-21, 19:15
+
Andrew Purtell 2012-02-18, 17:56
+
tsuna 2012-02-22, 20:20
+
Jeff Whiting 2012-02-23, 21:40
+
Stack 2012-02-23, 22:14