Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - modularizing trunk

Copy link to this message
modularizing trunk
Jesse Yates 2012-05-09, 01:31
Hi devs,

I'm just finishing up work on
modularization of hbase - and am planning to work with Stack to commit
the patch to trunk. If there are no objections, the planned refactor is on *May
23, 2012 (hbase hackathon).*

*What does this mean for you?*

If at the commit, you don't have any patches you will just pull the latest
version, import the projects into eclipse and move on with your life. It
may or may not be slightly more complex to import, depending on how you are
currently importing the project, but details on specific procedures will
come with the commit.

If you do have open patches, its a simple process to convert them over. You
will export your diff into a patch file (lets assume here its the standard
--no-prefix format). After pulling down the new version, you will just cd
into hbase-core and apply your patch. Unless you are modifying the
reference guide, hbase/bin scripts, hbase/conf files or the pom (please
avoid until after the switch as we will be changing everything), then the
patch _should_ apply cleanly. If you are modifying those files, you will
need to split up the patch file and apply the first half from hbase and the
second from hbase-core.

*What is actually happening in this patch?*

The initial plan for HBASE-4336 was to break out each of the different
pieces of hbase into their own module. However, in practice doing the
one-shot refactor is a huge pain and not really feasible with the current
project velocity. Instead, 4336 sets up the framework to enable much easier
modularization. If we want to pull out client stuff, then we merely need to
dis-entangle the code, add a few lines to the hbase-assembly project, and
you are off to the races (more or less - see
caveats around tests).

After the patch goes in, we will have three modules: hbase, hbase-core,

   - hbase is the parent and defines the general properties used by all the
   children, including managing dependency versions, plugin versions and
   general properties, and managing the docbook and site.
   - hbase-core is where all the code will go. It will still export a
   regular jar and a test jar and support the security profile (though
   activation will now be -Dsecurity, rather than -Psecurity)
   - hbase-assemble will package everything together into the final

In the future, packages like mapreduce, client, sever, wal, etc have been
discussed. The means easier testing, development and deployment.

*How can you help?

*1) Get as much of your code into the trunk before the refactor
2) Take a look at https://github.com/jyates/hbase/tree/refactor-modules and
comment on the recent patches there (this is pretty close to what the final
refactor will look like)
3) Start thinking about refactoring other modules out of hbase-core (or
even forking on github and doing it!)

Jesse Yates
Ted Yu 2012-05-09, 01:51
Stack 2012-05-09, 17:43
Matt Corgan 2012-05-09, 18:18
Stack 2012-05-09, 19:01
Andrew Purtell 2012-05-09, 19:06
Jesse Yates 2012-05-09, 20:27
Matt Corgan 2012-05-09, 20:34
Elliott Clark 2012-05-09, 21:18
Jesse Yates 2012-05-09, 22:22
Jesse Yates 2012-05-24, 06:27
Jesse Yates 2012-05-24, 19:29
Andrew Purtell 2012-05-24, 19:43
Stack 2012-05-25, 15:54