Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> adding constraints


+
Jesse Yates 2011-10-17, 17:45
+
Ted Yu 2011-10-17, 18:00
+
Jesse Yates 2011-10-17, 18:04
+
Ted Yu 2011-10-17, 18:10
+
Jesse Yates 2011-10-17, 18:27
+
lars hofhansl 2011-10-18, 05:00
+
Jesse Yates 2011-10-18, 05:24
+
Gary Helmling 2011-10-18, 07:43
+
Andrew Purtell 2011-10-18, 22:31
+
Andrew Purtell 2011-10-18, 22:42
+
Jesse Yates 2011-10-19, 00:49
Copy link to this message
-
Re: adding constraints
>>
>> Hmm, I wasn't really reading the two implementation options for
>> constraints as a choice between a "built-in" feature and CP based.
>>
>
> Either way it would be CP based, but the 'built-in' would just have some
> 'nice' ways of adding things. In short, its a question of adding a method to
> the HTD for addConstraint() to add a bunch of classes to be run by the
> 'constraint CP'.
>

I think we're on the same page here (just the details to work out).
But I think for most people on this list, saying "top level" or "built
in" feature would imply something not CP based, so we should be
careful about terminology.

>
> I feel like having the addConstraint() for a table is actually _less_
> complexity. Not necessarily from the overall system perspective certainly
> (you have to do a little abstraction and a couple more methods), but its not
> that much more as it all centered around the HTD.
>

For a single case, yes, this is simpler.  But it shifts complexity
from the exposed configuration into the HTD code.  What happens when
we have 20 such cases?  HTD starts to become a bit of a mess with
special casing for each.  I totally understand the motivation -- we
did something similar with table "owners" in the patch for HBASE-3025.
 But I'm starting to think we need to handle it differently there and
here to keep things scalable.

I think we need to invert this, so that CPs can take ownership for
adding their own configs to HTD, instead of making HTD take ownership
for all.  Something like:

HTableDescriptor htd = new HTableDescriptor(...);
Constraints.add(htd, MyConstraintImpl.class);
admin.createTable(htd);

I think this is the best way to keep the code extensibility scalable.

We'd have to work out how exactly this integrates with the HBase
shell.  But given that jruby gives us a dynamic language to work with,
we should be able to figure something out.  I think making the shell
more extensible is also an important part of this.  For HBASE-3025 we
needed to add some shell commands, and there's not really a "loadable"
way of doing so at the moment.

>
> What I'm concerned about is the configuration complexity - there are a ton
> of them and adding more starts to be crazy. HBase has already made some
> tradeoffs, but if we keep adding more and more configuration values, its
> going to be close to unusable to anyone that doesn't have serious knowledge
> about the system and how to configure it.
>

I completely agree that security has a long way to go here.  Some
configuration has to be there -- we need to know the principals for
the various services, keytab files for logins -- but the rest of the
config for the loadable security bits should really be just a single
setting.  I totally agree with the vision here.  We'll get there.

>
> Ok, clearly the main thread through all of this is I would like to make it
> easier to load/unload features.
>
> Constraints was something (a) I thought hbase could use, (b) would be doable
> pretty easily with CPs, and (c) would put us down the path of making hbase
> easier to run/setup for users. The latter goes for security, constraints,
> and other new/existing features.
>

Agree with all of this, and I appreciate that you're looking at
improving this stuff.  Configuration and operability is a critical
part of the user experience and we have a long way to go in
streamlining it.

--gh
+
Jesse Yates 2011-10-19, 02:41
+
Gary Helmling 2011-10-19, 18:18
+
Jesse Yates 2011-10-19, 18:29