Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - modular build and pluggable rpc


Copy link to this message
-
Re: modular build and pluggable rpc
Gary Helmling 2011-05-27, 20:20
+1

Using maven modules would also allow us to have a minimal hbase-client.jar,
which is periodically requested on the mailing lists.

As we move to support more versions of Hadoop, being able to have separate
modules built against each version seems saner than continuing to extend the
current reflection based approaches, which can be brittle.

Since the security work has all been developed as loadable components,
making it a separate module would make perfect sense as a means of
integration.  Security can then be built against secure Hadoop for those who
care, while not impacting core HBase.  Same goes for supporting changes
across Hadoop 0.21, 0.22, trunk...

I agree we should branch 0.92 first, then get trunk modularized as soon as
possible.

If we all agree on that, I'm happy to help the modularization effort (with
limited maven skills), and will start posting security patches for review as
soon as we have the setup in place to support it.

--gh
On Fri, May 27, 2011 at 12:49 PM, Andrew Purtell <[EMAIL PROTECTED]>wrote:

> From IRC:
>
> apurtell        i propose we take the build modular as early as possible to
> deal with multiple platform targets
> apurtell        secure vs nonsecure
> apurtell        0.20 vs 0.22 vs trunk
> apurtell        i understand the maintenence issues with multiple rpc
> engines, for example, but a lot of reflection twistiness is going to be
> worse
> apurtell        i propose we take up esammer on his offer
> apurtell        so branch 0.92 asap, get trunk modular and working against
> multiple platform targets
> apurtell        especially if we're going to see rpc changes coming from
> downstream projects...
> apurtell        also what about supporting secure and nonsecure clients
> with the same deployment?
> apurtell        zookeeper does this
> apurtell        so that is selectable rpc engine per connection, with a
> negotiation
> apurtell        we don't have or want to be crazy about it but a rolling
> upgrade should be possible if for example we are taking in a new rpc from fb
> (?) or cloudera (avro based?)
> apurtell        also looks like hlog modules for 0.20 vs 0.22 and
> successors
> apurtell        i think over time we can roadmap the rpc engines, if we
> have multiple, by deprecation
> apurtell        now that we're on the edge of supporting both 0.20 and
> 0.22, and secure vs nonsecure, let's get it as manageable as possible right
> away
>
> St^Ack_         apurtell: +1
>
> apurtell        also i think there is some interest in async rpc engine
>
> St^Ack_         we should stick this up on dev i'd say
>
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>