Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> HBase wire compatibility


Copy link to this message
-
Re: HBase wire compatibility
Good writeup, Jimmy (was away for a few days due to an event in my family)
 
Some quick questions - Has there been any thoughts on the plan to use HBASE-5394. Are we going to make the hbase protocols (like HRegionInterface) protobuf aware?

In Hadoop, I have seen the following:
1. In HDFS, the protocol definitions are not changed (like org.apache.hadoop.hdfs.protocol.ClientProtocol). Instead there are translators that are defined that implement the mapping of protobuf datastructures to application-level datastructures and vice versa (for example, have a look at ClientNamenodeProtocolTranslatorPB and ClientNamenodeProtocolServerSideTranslatorPB in the package org.apache.hadoop.hdfs.protocolPB).
2. In Yarn (MRV2), all protocol definitions are written in PB
 
Since the base RPC still uses writables for payload encoding, a translation happens when the protobuf objects are sent/received (as an example look at org.apache.hadoop.ipc.ProtobufRpcEngine; classes RpcRequestWritable and RpcResponseWritable).
 
What does the HBase community think about the above?
On Feb 13, 2012, at 1:02 PM, Jimmy Xiang wrote:

> I posted the proposal on wiki:
>
> http://wiki.apache.org/hadoop/Hbase/HBaseWireCompatibility
>
> Thanks,
> Jimmy
>
> On Mon, Feb 13, 2012 at 11:03 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Can you post on wiki ?
>>
>> Attachment stripped.
>>
>> On Mon, Feb 13, 2012 at 11:01 AM, Jimmy Xiang <[EMAIL PROTECTED]> wrote:
>>
>>> Hello,
>>>
>>> As HBase installation base is getting bigger, we are ready to work on the
>>> wire compatibility issue.
>>> The goal is to make HBase easier for operators to upgrade, while it is
>>> also easier for developers to
>>> enhance, re-architect if necessary.
>>>
>>> The attached is a proposal we came up.  We'd like to start with two
>> phases:
>>>
>>> Phase 1: Compatibility between client applications and HBase clusters
>>> Phase 2: HBase cluster rolling upgrade within same major version
>>>
>>> Could you please review?
>>>
>>> Thanks,
>>> Jimmy
>>>
>>