Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - hbase 0.94.0


Copy link to this message
-
Re: hbase 0.94.0
Devaraj Das 2012-01-31, 07:53

On Jan 30, 2012, at 11:39 AM, Jonathan Hsieh wrote:

> Nicolas,
>
> For point 2 and 3, there definitely is a group of us that have been
> informally discussing long term wire and format compatibility in
> conversations.  There are some documents that should be coming out soon to
> the lists from these chats. The initial goal of this some of is to define a
> vocabulary, to try to scope and phase-in changes necessary to make this
> happen (possibly parts in 0.94, 0.96, 0.98...), and to define possible
> goals and non-goals.
>
> My hope is that a thought-out first cut will come from Jimmy and I in the
> next week or so so that we start having discussion to reach a consensus and
> then start implementing.
>
> Some folks from another group (I'll let them announce themselves) should be
> producing a doc focused more on the rpc portion of this soon as well.
>

Thanks for the hint, Jon (*smile*)

BTW I have opened - https://issues.apache.org/jira/browse/HBASE-5305 / HBASE-5306 to keep track of the issues..

We will upload some docs/discussion-points there soon.
> Jon.
>
> On Mon, Jan 30, 2012 at 11:21 AM, Nicolas Spiegelberg
> <[EMAIL PROTECTED]>wrote:
>
>> Currently, we at Facebook are developing mostly on 89 but also doing some
>> significant exploratory work on trunk.  I think that most of our
>> development will continue to be done on 89 in the near future.  We plan to
>> release some lower-risk projects on 94.  However, we won't even entertain
>> a wide-scale upgrade strategy until:
>>
>> 1) The trunk has more master work.  Specifically 89-master related
>> features that we implemented for our grid architecture.  It sounds like
>> most of those features are rising in prominence in the Open Source
>> community as well (Ted from eBay & Todd from Cloudera, in particular).
>> Hopefully, we can collaborate on porting/implementation.
>> 2) Client/Server compatibility.  Even if we upgraded the servers to
>> 94/96/whatever, we still will have a lot of clients running 89 & we can't
>> simply stop/start all of them at the same time.  We need 89/trunk RPC
>> compat for client/server.
>> 3) Durable Server Upgrade.  It does bother me that people are very quick
>> to consider removing both RPC & persistent data compatibility from 90 to
>> trunk (a branch that we're still actively releasing minor upgrades for).
>> We will need the ability to mutate the HDFS & ZK persistent data from the
>> 89 format to trunk.  Specifically, work like HFileV1 reader is critical
>> for analysis if the upgrade script fails and we need to debug.
>>
>> Instead of discussing what classes we can throw away, can we instead think
>> about a strategy for long-term, cross-version compatibility that minimally
>> hurts trunk development?  Is HFileV1's presence severely hindering another
>> feature?
>>
>> On 1/27/12 6:11 PM, "Jonathan Hsieh" <[EMAIL PROTECTED]> wrote:
>>
>>> Lars, Matt,
>>>
>>> I like Matt's suggestion -- as long as it is not a burden let's keep it
>>> in.
>>> If it becomes one, let's do what is best for the project.
>>>
>>> If Cloudera needs to do the upgrade path, we'll provide one.  If it
>>> doesn't
>>> go to the apache repo, we'll most likely open source it on github or
>>> something, or "keep it on the jira" like some of the repair ruby scripts.
>>>
>>> Question -- to the Trend/SalesForce/FB folks can you talk about your plans
>>> for upgrades?  Are you guys moving from 0.89/0.90ish to 0.92 already?  or
>>> are you guys going to have to do the direct upgrade to 0.94 from 0.90 land
>>> as well?
>>>
>>> Jon.
>>>
>>>
>>> On Fri, Jan 27, 2012 at 12:06 PM, Matt Corgan <[EMAIL PROTECTED]>
>> wrote:
>>>
>>>> The reason I brought it up is now that Mikhail committed the data block
>>>> encoding I was going to take a stab at adding the prefix trie encoding I
>>>> was working on this past summer.  My plan is to first make a minimally
>>>> invasive patch to prove correctness.  But, after that there will
>>>> probably
>>>> be some big performance gains to be had from reworking some of the