Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - adding constraints


Copy link to this message
-
Re: adding constraints
Jesse Yates 2011-10-19, 02:41
On Tue, Oct 18, 2011 at 7:10 PM, Gary Helmling <[EMAIL PROTECTED]> wrote:

> >>
> >> Hmm, I wasn't really reading the two implementation options for
> >> constraints as a choice between a "built-in" feature and CP based.
> >>
> >
> > Either way it would be CP based, but the 'built-in' would just have some
> > 'nice' ways of adding things. In short, its a question of adding a method
> to
> > the HTD for addConstraint() to add a bunch of classes to be run by the
> > 'constraint CP'.
> >
>
> I think we're on the same page here (just the details to work out).
> But I think for most people on this list, saying "top level" or "built
> in" feature would imply something not CP based, so we should be
> careful about terminology.
>

Agreed.
>
> >
> > I feel like having the addConstraint() for a table is actually _less_
> > complexity. Not necessarily from the overall system perspective certainly
> > (you have to do a little abstraction and a couple more methods), but its
> not
> > that much more as it all centered around the HTD.
> >
>
> For a single case, yes, this is simpler.  But it shifts complexity
> from the exposed configuration into the HTD code.  What happens when
> we have 20 such cases?  HTD starts to become a bit of a mess with
> special casing for each.  I totally understand the motivation -- we
> did something similar with table "owners" in the patch for HBASE-3025.
>  But I'm starting to think we need to handle it differently there and
> here to keep things scalable.
>

Yeah, this can start to be a bit of a mess.
>
> I think we need to invert this, so that CPs can take ownership for
> adding their own configs to HTD, instead of making HTD take ownership
> for all.  Something like:
>
> HTableDescriptor htd = new HTableDescriptor(...);
> Constraints.add(htd, MyConstraintImpl.class);
> admin.createTable(htd);
>

+1 I like the idea. It also feels very 'hadoopy' (eg. input/output formats)
>
> I think this is the best way to keep the code extensibility scalable.
>
> We'd have to work out how exactly this integrates with the HBase
> shell.  But given that jruby gives us a dynamic language to work with,
> we should be able to figure something out.  I think making the shell
> more extensible is also an important part of this.  For HBASE-3025 we
> needed to add some shell commands, and there's not really a "loadable"
> way of doing so at the moment.
>

It will be interesting to see how that feature shakes out

Do you think you can put the core what we are agreeing on into 4605?  I want
to make sure we don't lose any of your comments

Thanks,
Jesse