Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - updating shell functionality


Copy link to this message
-
updating shell functionality
Jesse Yates 2011-11-22, 18:34
Hey all,

I'm trying to take some time to figure out the right way to do support to
future 'pluggable' features/extensions to modifying tables. Specially, I'm
thinking about the 'alter' command, but in the future, this model could be
applied to other facets of the shell.

 In the easiest, and hackiest) way, we could just take in the full class
name to call and then lookup the java class, but I like Gary's comment
from HBASE-4605
(Constraints) <https://issues.apache.org/jira/browse/HBASE-4605>:

"3. provide a extensions dir for the shell:

   - extensions drop a simple jruby scriptlet in a file in the dir (say
   lib/ruby/hbase/ext)
   - the scriptlet does some simple registration of the available
   methods/commands
   - the shell code loads all files"

In doing this a 'clean' way, we could also do a lot in making the shell
easier to use. (Stick with me, its gets a little verbose here).

What I'm proposing is that instead of altering tables like this(for setting
arbitrary keys, but this could be things like MAX_FILESIZE, etc):

 alter 'table', METHOD=>'table_attr', "key1" => "v1"

We could actually have the use make explicit method calls (which should
give auto-complete in irb) on an object you get from alter. Then you can do
something like this:

 hbase> alter('table').table_attr "key1" => "v1"

in the simple case. And we allow a lot of the basic functionality you
already get with Java HTD.

However, this becomes interesting when we want to add more features. For
instance, with constraints, it would look something like:

 hbase> alter('table').addConstraint('com.my.Constraint')

What's even cooler is, since we can reopen ruby classes and add
functionality - for instance when we load all the files in the
lib/ruby/hbase/ext - we are actually open up the object returned by alter
(maybe something like AlterTable class?) and add methods. This means we
also get auto-complete when doing an alteration on a table.

This is then a really extensible way to add new functionality and cleanup
older. The ext scripts just open up the actual classes they need to modify
to add the method functionality, without all the mucking around with
strings that we are doing now.

For instance, adding a coprocessor can go from:

 hbase> alter 't1', METHOD =>
'table_att''coprocessor'=>'hdfs:///foo.jar|com.foo.FooRegionObserver|1001|arg1=1,arg2=2
to now something like:

 hbase> alter('t1').addCoprocessor 'com.foo.FooRegionObserver',
'hdfs:///foo.jar', '1001' {'arg1'=>'1','arg2'=>'2'}
Or slightly more verbosely:

 hbase> alter('t1').addCoprocessor('com.foo.FooRegionObserver',
'hdfs:///foo.jar', '1001' {'arg1'=>'1','arg2'=>'2'})
This approach might also be something we can extend to much of the current
shell functionality while still retaining much of the current code. I think
moving this this kind of approach applies anywhere we would need to do
chaining of invocations. Clearly, it depends on the situation, but
leveraging OOP, rather than switching on string, makes a lot of sense (at
least to me :)

What does everyone think of this approach? Am I missing anything
technically (haven't really played with JRuby before) or historically? Or
maybe people have other preferences?

Thanks!

-- Jesse
-------------------
Jesse Yates
240-888-2200
@jesse_yates