Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Proposed feature branch for HBase security

Copy link to this message
Re: Proposed feature branch for HBase security
Gary Helmling 2010-07-06, 20:05
> > * Hadoop RPC version rolled from 3 to 4 (apparently 0.20-append also does
> > this!)
> >
> Can you explain further which version you're talking about here? Our HBase
> IPC is already wire-incompatible with Hadoop IPC, so the version numbers
> needn't match up, right?
> Correct, this is really only a complication in producing a combined Hadoop
0.20 append+security branch (see Andy Purtell's recent email), so not really
relevant here.  For the HBase RPC update, we can of course handle the
versioning however we want.
> > * various bits in the updated HBaseClient, HBaseServer, etc. now depend
> on
> > the security implementation, so building and running on top of non-secure
> > Hadoop will not be possible.
> >
> >
> Which classes do we depend on? Can we copy-paste those over into our tree?
> It's more of a maintenance pain in some ways, but in other ways it allows
> us
> to fix bugs, etc, without waiting on Hadoop releases.
The main new classes I brought over are:

org.apache.hadoop.security.SaslRpcClient ->
org.apache.hadoop.security.SaslRpcServer ->

which depend on:


other RPC changes depend on:

o.a.hadoop.security.UserGroupInformation (incompatible interface changes)

which further expands into the org.apache.hadoop.security package and

My goal here has really just been to get bootstrapped on top of secure
Hadoop, so I've been opting for the minimal changes/duplication necessary.
We could duplicate more and strip out unnecessary bits to fully insulate our
changes and achieve cross Hadoop version compatibility.  Cross Hadoop
version compatibility will be critical before any of this could go into
trunk, but I don't think anything here would restrict taking that option
later on.  I'm just trying to defer that decision/work in case something
better comes along in the interim.  :)  In any case, secure RPC changes
wouldn't really have any value in themselves without a secure HBase
implementation, so this seems best fitted as a first step in a security
> > I'd like to post the diff on review.hbase.org for more review and
> > feedback,
> > but that begs the question of where the changes should go?
> >
> >
> I can set up review.hbase.org to fetch from your github repo, so that
> diffs
> against your github branch will upload properly. Is that useful?
At this point, I think it would help along to have some concrete changes to
refer to, so I'll follow up by posting to rb.  Whether or not the github
setup would be useful I think depends on consensus for basing security
development in an svn feature branch vs. externally (ie github)?

Thanks for the feedback and thoughts.