Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> hbase 0.94.0

Currently, we at Facebook are developing mostly on 89 but also doing some
significant exploratory work on trunk.  I think that most of our
development will continue to be done on 89 in the near future.  We plan to
release some lower-risk projects on 94.  However, we won't even entertain
a wide-scale upgrade strategy until:

1) The trunk has more master work.  Specifically 89-master related
features that we implemented for our grid architecture.  It sounds like
most of those features are rising in prominence in the Open Source
community as well (Ted from eBay & Todd from Cloudera, in particular).
Hopefully, we can collaborate on porting/implementation.
2) Client/Server compatibility.  Even if we upgraded the servers to
94/96/whatever, we still will have a lot of clients running 89 & we can't
simply stop/start all of them at the same time.  We need 89/trunk RPC
compat for client/server.
3) Durable Server Upgrade.  It does bother me that people are very quick
to consider removing both RPC & persistent data compatibility from 90 to
trunk (a branch that we're still actively releasing minor upgrades for).
We will need the ability to mutate the HDFS & ZK persistent data from the
89 format to trunk.  Specifically, work like HFileV1 reader is critical
for analysis if the upgrade script fails and we need to debug.

Instead of discussing what classes we can throw away, can we instead think
about a strategy for long-term, cross-version compatibility that minimally
hurts trunk development?  Is HFileV1's presence severely hindering another

On 1/27/12 6:11 PM, "Jonathan Hsieh" <[EMAIL PROTECTED]> wrote:

>Lars, Matt,
>I like Matt's suggestion -- as long as it is not a burden let's keep it
> If it becomes one, let's do what is best for the project.
>If Cloudera needs to do the upgrade path, we'll provide one.  If it
>go to the apache repo, we'll most likely open source it on github or
>something, or "keep it on the jira" like some of the repair ruby scripts.
>Question -- to the Trend/SalesForce/FB folks can you talk about your plans
>for upgrades?  Are you guys moving from 0.89/0.90ish to 0.92 already?  or
>are you guys going to have to do the direct upgrade to 0.94 from 0.90 land
>as well?
>On Fri, Jan 27, 2012 at 12:06 PM, Matt Corgan <[EMAIL PROTECTED]> wrote:
>> The reason I brought it up is now that Mikhail committed the data block
>> encoding I was going to take a stab at adding the prefix trie encoding I
>> was working on this past summer.  My plan is to first make a minimally
>> invasive patch to prove correctness.  But, after that there will
>> be some big performance gains to be had from reworking some of the
>> like KeyValueScanner which I would not have the courage to get working
>> v1.
>> So, that was why I asked, but all of that is still more hypothetical
>> real, and I don't even know if the first part will be done before
>> .94 at the end of February.  Makes sense to me to not delete v1 until
>> there's a good reason to, which it doesn't look like we have yet.  If I
>> to the point where v1 is halting progress then we can reevaluate based
>> more specific issues.  Maybe none of the prefix trie will even make
>> ..sent from my phone
>> On Jan 27, 2012 1:55 PM, "lars hofhansl" <[EMAIL PROTECTED]> wrote:
>>>  Hey Jon,
>>> understood. Makes 0.94 hard, though. If we decide now to have a 0.90 to
>>> 0.94 upgrade path and then timing does not work out and nobody signs
>>>up for
>>> the testing because it 0.92 is more convenient we'd have gone through
>>> for nothing.
>>> So... Thinking about this more I am -1 on supporting an official
>>> path other then from one release to the next.
>>> That said, we do not have to break things intentionally.
>>> I'm fine pushing HBASE-2600 and HFile v1 removal out of 0.94... As long
>>> as we won't havethe same argument for 0.96 :)
>>> And I am not aware of any file compatibility issues.