Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Re: Upcoming merge of snapshots branch into trunk. (HBASE-6055 and HABSE-7290)


+
Jonathan Hsieh 2013-01-11, 13:33
+
Andrew Purtell 2013-01-11, 16:16
Copy link to this message
-
Upcoming merge of snapshots branch into trunk. (HBASE-6055 and HABSE-7290)
Hey Folks,

Matteo, Jesse and I seem to be getting to the point where have core
functionality for offline snapshots (disable table, snapshot) and online
snapshot (snapshot an enabled table) committed, did another rev solidifying
file layout, and have been steadily knocking off blocking and non-blocking
follow-on subtasks.

As a heads up, I think we are going to start considering a merge of the
snapshots branch to trunk. We agreed early on that we'd want 3 +1s before
the merge occurred.  Matteo, Jesse, and I worked on the core and all
committers now so we could technically satisfy this, but I'd really like to
have at least 1 reviewer look at the whole thing who didn't write parts of
it. :)  We'll also try to do another update on the design docs to help the
reviews.  Please consider taking a look at the branches and let us know.

We're going to be spending this week hardening and testing are reasonable
scale to shake out more problems, so this may happen starting some time
next week.

Aside from asking for reviews, there are a few outstanding questions we'd
love to get your feedback on:
HBASE-7471 - default configuration so that snapshots are available by
default?
HBASE-7360 - do we backport offline snapshots to the apache hbase 0.94 line?

So where is the code and jiras? Currently, there are two branches -- an
offline snapshot only branch (HBASE-6055) here (
https://github.com/jyates/hbase/tree/snapshots aka
https://github.com/jmhsieh/hbase/tree/hbase-6055) and an offline + online
one (HBASE-7290) here
(https://github.com/jmhsieh/hbase/tree/snapshots).  Currently the
difference between the two are 3-4 patches (they are fairly substantial but
primarily additive).  We were being conservative initially and had hoped
that we could of done an offline merge earlier (and possibly merge with 94)
and then a second round when the online snapshot was more robust.  If our
testing goes well this week, for trunk I'd lean more towards just doing one
merge pass with the offline+online branch.

Here's my current thoughts on the strategy and mechanics of the merge
(hopefully sometime next week).  Currently, our branches are based on a
12/18 version of trunk. We're going to start doing merges of trunk into the
snapshot branch (to get all trunk patches into the snapshots).   I'll also
post a mega patch on review board after.  After we get all the reviews, we
could go a couple of ways, and would like your thoughts:

1) just commit the consolidated mega patch to trunk.  (every trunk step
should compile, but we lose blame information)
2) rebase each patch and then a fix up patch at the end (yuck -- may have
broken intermediate steps, but keeps blame info)
3) create a branch in svn (we've been in github) and, after we do an svn
merge of the snapshot branch into trunk. (more work, but each commit in
each branch should compile, keep blame information).

#1 would look like this:

   SS  SS is a single mega patch commit.
   |
S4 |   A merge of trunk into the snapshot branch (could be multiple)
| \|
|  T3  Trunk patch that when in while snapshots under dev.
S3 |
|  T2
S2 |
S1 |   Snapshot branch patches
  \|
   T1  The 12/18 point where snapshots was last rebased.
   ...

#3 would be a bit of work but I think would be best in the end.   I think
the history would roughly look like this: (S is a snapshot commit, T is
trunk)

   T4  Final merge commit into trunk. should be the same as S4
  /|
S4 |   A merge of trunk into the snapshot branch (could be multiple)
| \|
|  T3  Trunk patch that when in while snapshots under dev.
S3 |
|  T2
S2 |
S1 |   Snapshot branch patches
  \|
   T1  The 12/18 point where snapshots was last rebased.
   ...

Comments, concerns?

Thanks,
Jon.

--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// [EMAIL PROTECTED]
+
Stack 2013-01-08, 07:44
+
Ted Yu 2013-01-07, 18:38
+
Jonathan Hsieh 2013-01-07, 18:58
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB