-
Re: Patches for the offline and online snapshots branches.Jonathan Hsieh 2012-12-29, 08:02
Stack, Lars,
tl;dr:
* yeah, with dev-branch we need to be more liberal with commit.
* for trunk merge, we agreed 3x +1s required, regardless of +1's on
individual patches on the branch.
* Reviews? please take a look at HBASE-6864, HBASE-7321 next and
subtasks of HBASE-6055.
* Considering remerging offline-snapshot and online snapshot branch.
==
When we when we went on to the branch approach, it was with the intent
of being a bit more liberal, and also agreed that 3x +1's were
required to merge back onto trunk. Many of the early patches were
plus +1'ed with several caveats so that we could come back and fix
them before merging to trunk.
While working alone or while tests were consistently passing,
maintaining a stack of patches was manageable (Jesse had a bunch for a
while, and I've I had about 5-6 for a month, updating every day or
so). It wasn't too bad until the past few days when Matteo and I have
been stepping on each others toes while we've been fighting bugs and
failing tests -- this is where keeping all these stacks became overly
burdensome. It is difficult to isolate problems when the underlaying
code changes frequently, and it is hard to have a solid point to stand
upon when new bugs pop up unless code is committed and not changing
underneath. Rebases to trunk are necessary but also have caused some
problems.
I committed the refactored core of the offline snaphots -- the one
that I said I'd commit (HBASE-7208 - it was on v8) along with a few
renames/package moves that are fairly trivial (HBASE-7207, HBASE-7454,
HBASE-7452). Our initial plan was to get the offline branch (along
with restore and clone operations) into shape and merge that with
trunk and into 0.96. We need to get a few more guard rails patches in
and do more testing of restored tables such as HBASE-7419, HBASE-7294,
HBASE-7352 before we consider merging to trunk. There are also some
clean failure recovery patches that we should get in but that I'm less
adamant about before we merge (HBASE-7389/HBASE-7385).
If you are looking for code to review look at the patch available list
on the sub tasks of the offline snapshot jira.
https://issues.apache.org/jira/browse/HBASE-6055
I've also started another branch with the commit of HBASE-7212 (global
procedure also on v8). It's umbrella jira is here:
https://issues.apache.org/jira/browse/HBASE-7290
The plan here was to get one of the online snapshot flavors in to the
online snapshot branch and then merge it to trunk (possibly after the
first 0.96 release). I need to post updates for these, and would
like reviews for them, but the bare minimum to reach this would be to
get these patches in.
HBASE-6864 - online snapshot scaffolding
HBASE-7321 - simple flush online snapshot
We may want to revisit offline-online split since the simplest online
snapshots actually less troublesome (everything happens and then an
atomic rename completes taking the snapshot) than failure recovery of
clone and restore (we initially didn't do an atomic rename when
restoring table data, and even with that we need meta modifications to
atomically happen with the fs changes)
Jon.
On Fri, Dec 28, 2012 at 10:44 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> I think for feature branches we can be quite bit more liberal about applying patches.
> Of course then the question is how to go about the eventual merge back into trunk.
>
> If all patches in the branch were reviewed and +1'ed, the merge in theory would not need to reviewed again.
> Is what you are shooting for, Jon?
>
>
> -- Lars
>
>
>
> ________________________________
> From: Stack <[EMAIL PROTECTED]>
> To: HBase Dev List <[EMAIL PROTECTED]>
> Sent: Friday, December 28, 2012 10:21 PM
> Subject: Re: Patches for the offline and online snapshots branches.
>
> What can we review to help Jon? What issue numbers?
> St.Ack
>
>
> On Thu, Dec 27, 2012 at 2:02 PM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>
>> Hey all,
>>
>> I'm going to post one more rev for these and then plan on committing
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]
tl;dr:
* yeah, with dev-branch we need to be more liberal with commit.
* for trunk merge, we agreed 3x +1s required, regardless of +1's on
individual patches on the branch.
* Reviews? please take a look at HBASE-6864, HBASE-7321 next and
subtasks of HBASE-6055.
* Considering remerging offline-snapshot and online snapshot branch.
==
When we when we went on to the branch approach, it was with the intent
of being a bit more liberal, and also agreed that 3x +1's were
required to merge back onto trunk. Many of the early patches were
plus +1'ed with several caveats so that we could come back and fix
them before merging to trunk.
While working alone or while tests were consistently passing,
maintaining a stack of patches was manageable (Jesse had a bunch for a
while, and I've I had about 5-6 for a month, updating every day or
so). It wasn't too bad until the past few days when Matteo and I have
been stepping on each others toes while we've been fighting bugs and
failing tests -- this is where keeping all these stacks became overly
burdensome. It is difficult to isolate problems when the underlaying
code changes frequently, and it is hard to have a solid point to stand
upon when new bugs pop up unless code is committed and not changing
underneath. Rebases to trunk are necessary but also have caused some
problems.
I committed the refactored core of the offline snaphots -- the one
that I said I'd commit (HBASE-7208 - it was on v8) along with a few
renames/package moves that are fairly trivial (HBASE-7207, HBASE-7454,
HBASE-7452). Our initial plan was to get the offline branch (along
with restore and clone operations) into shape and merge that with
trunk and into 0.96. We need to get a few more guard rails patches in
and do more testing of restored tables such as HBASE-7419, HBASE-7294,
HBASE-7352 before we consider merging to trunk. There are also some
clean failure recovery patches that we should get in but that I'm less
adamant about before we merge (HBASE-7389/HBASE-7385).
If you are looking for code to review look at the patch available list
on the sub tasks of the offline snapshot jira.
https://issues.apache.org/jira/browse/HBASE-6055
I've also started another branch with the commit of HBASE-7212 (global
procedure also on v8). It's umbrella jira is here:
https://issues.apache.org/jira/browse/HBASE-7290
The plan here was to get one of the online snapshot flavors in to the
online snapshot branch and then merge it to trunk (possibly after the
first 0.96 release). I need to post updates for these, and would
like reviews for them, but the bare minimum to reach this would be to
get these patches in.
HBASE-6864 - online snapshot scaffolding
HBASE-7321 - simple flush online snapshot
We may want to revisit offline-online split since the simplest online
snapshots actually less troublesome (everything happens and then an
atomic rename completes taking the snapshot) than failure recovery of
clone and restore (we initially didn't do an atomic rename when
restoring table data, and even with that we need meta modifications to
atomically happen with the fs changes)
Jon.
On Fri, Dec 28, 2012 at 10:44 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> I think for feature branches we can be quite bit more liberal about applying patches.
> Of course then the question is how to go about the eventual merge back into trunk.
>
> If all patches in the branch were reviewed and +1'ed, the merge in theory would not need to reviewed again.
> Is what you are shooting for, Jon?
>
>
> -- Lars
>
>
>
> ________________________________
> From: Stack <[EMAIL PROTECTED]>
> To: HBase Dev List <[EMAIL PROTECTED]>
> Sent: Friday, December 28, 2012 10:21 PM
> Subject: Re: Patches for the offline and online snapshots branches.
>
> What can we review to help Jon? What issue numbers?
> St.Ack
>
>
> On Thu, Dec 27, 2012 at 2:02 PM, Jonathan Hsieh <[EMAIL PROTECTED]> wrote:
>
>> Hey all,
>>
>> I'm going to post one more rev for these and then plan on committing
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]
