Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - [PROPOSAL] HBASE-10070 branch


Copy link to this message
-
[PROPOSAL] HBASE-10070 branch
Enis Söztutar 2014-01-15, 08:44
Hi,

I just wanted to give some updates on the HBASE-10070 efforts from the
technical side, and development side, and propose a branch.

>From the technical side:
The changes for region replicas phase 1 are becoming more mature and
stable, and most of the "base" changes are starting to become good
candidates for review. The code has been rebased to trunk, and the main
working repo has been moved to the HBASE-10070 branch at
https://github.com/enis/hbase/tree/hbase-10070.

An overview of the changes that is working include:
 - HRegionInfo & MetaReader & MetaEditor changes for support region replicas
 - HTableDescriptor changes and shell changes for supporting
REGION_REPLICATION
 - WebUI changes to display whether a region is a replica or not
 - AssignmentManager changes coupled with RegionStates & Master changes to
create and assign replicas, alter table, enable table, etc support.
 - Fixed hbck to work with replicas
 - A Consistency API from client side together with shell support
 - Load Balancer changes for region replicas for replica placement
 - Region and RegionServer changes for opening region replicas, and
refreshing store files
 - Client side changes for RPC failover support for eventual consistent
gets
 - End to end test mentioned in
https://issues.apache.org/jira/browse/HBASE-10070?focusedCommentId=13849978&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13849978

These are some of the remaining things that we are currently working on:
 - RPC failover support for multi-gets
 - RPC failover support for scans
 - RPC cancellation
 - Ability to refresh the client's knowledge about replica location/count
changes
 - Integration tests
 - General hardening
 - Perf tests

Development side:
As discussed in the issue design doc
https://issues.apache.org/jira/secure/attachment/12616659/HighAvailabilityDesignforreadsApachedoc.pdf"Apache
code development process" section, at this time we would like to
propose:
 (1) Creation of HBASE-10070 branch in svn which will be a fork of trunk as
of the date branch is created. All of the target authors (me, Devaraj,
Nicolas, Sergey) are already committers. I do not remember whether our
bylaws require votes on creating branches.
 (2) The branch will only contain commits that have been reviewed and +1'ed
from 2 other committers other than the patch author. Every commit in this
branch will have a single patch (maybe with unforeseen addendums) and and
associated jira which is a subtask of HBASE-10070.
 (3) We will use the branch HBASE-10070 hosted at my github repo
https://github.com/enis/hbase/tree/hbase-10070 as a working branch with
semi-dirty history and "this branch might eat your hard drive" guarantees.
 (4) All code contributions / review will be welcome as always. I can give
you push perms to the github branch if you are interested in contributing.
 (5) Once we have HBASE-10070 Phase 1 tasks done (as described in the doc),
we will put up a VOTE to merge the branch in. We will require 3 +1's for
the merge in. If we can get early reviews the merge vote will be much less
pain since the branch will be in a clean state and there have been reviews
per patch. We might need a final rebase, but that should not cause major
work I imagine.

We are hoping this will be a nice way to develop and deliver the feature to
the trunk, but as always all suggestions, comments welcome.

Enis