Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Question on Coprocessors and Atomicity


Copy link to this message
-
Re: Question on Coprocessors and Atomicity
Jesse Yates 2011-12-04, 01:26
The row lock stuff was just if you are really concerned with making sure
the atomicity works (though I don't see any reason it wouldn't). Since all
the puts are going to be filtered through the coprocessor (the same
instance in fact), in order and the CP is going to block the Put from
completing (Because the CP is going to be doing the check in prePut), then
you can be sure you have atomic access to that put. Rows are locked on
update anyways, so the lock there is implicit - you would want commonly
want to use the row lock when doing changes to other rows to ensure
entity/referential integrity.

Keep in mind that you will always see a consistent view and since there is
only one CP running on one regionserver accesing that one row, you are
going to get the atomicity - the single thread is should never cause
conflicts with itself under normal operation. Failure scenarios could
theoretically be a little weird, if some puts fail for a given use case
that is dependent on that previous writes (but this is not yours).

Quick soln - write a CP to check the single row (blocking the put). If you
are looking at multiple rows, then you need to start worrying about complex
locking situations, but right now that should be fine.

Right soln (at least trying not to overthink it) - incorporate local table
access into constraints and then just impl a constraint.

-Jesse
On Sat, Dec 3, 2011 at 4:54 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:

> Just so my question is clear ... everything I'm suggesting is in the
> context of a single row (not cross row / table). - so, yes, I'm
> guessing obtaining a RowLock on the region side during preCheckAndPut
> / postCheckAndPut would certainly work. Which was why I was asking
> whether the pre/postCheckAndPut obtains the row lock or whether the
> row lock is only obtained within checkAndPut.
>
> Let's say the coprocessor takes a rowlock in preCheckAndPut ... will
> that even work? i.e. can the same rowlock be inherited by the
> checkAndPut api within that thread's context? Or will preCheckAndPut
> have to release the lock so that checkAndPut can take it (which won't
> work for my case, as it has to be atomic between the preCheck and
> Put.)
>
> Thanks for pointing me to the Constraints functionality - I'll take a
> look at whether it could potentially work.
> --Suraj
>
> On Sat, Dec 3, 2011 at 10:25 AM, Jesse Yates <[EMAIL PROTECTED]>
> wrote:
> > I think the feature you are looking for is a Constraint. Currently they
> are
> > being added to 0.94 in
> > HBASE-4605<https://issues.apache.org/jira/browse/HBASE-4605>;
> > they are almost ready to be rolled in, and backporting to 0.92 is
> > definitely doable.
> >
> > However, Constraints aren't going to be quite flexible enough to
> > efficiently support what you are describing. For instance, with a
> > constraint, you are ideally just checking the put value against some
> simple
> > constraint (never over 10 or always an integer), but looking at the
> current
> > state of the table before allowing the put would currently require
> creating
> > a full blown connection to the local table through another HTable.
> >
> > In the short term, you could write a simple coprocessor to do this
> checking
> > and then move over to constraints (which are a simpler, more flexible,
> way
> > of doing this) when the necessary features have been added.
> >
> > It is worth discussing if it makes sense to have access to the local
> region
> > through a constraint, though that breaks the idea a little bit, it would
> > certainly be useful and not overly wasteful in terms of runtime.
> >
> > Supposing the feature would be added to talk to the local table, and
> since
> > the puts are going to be serialized on the regionserver (at least to that
> > single row you are trying to update), you will never get a situation
> where
> > the value added is over the threshold. If you were really worried about
> the
> > atomicity of the operation, then when doing the put, first get the

Jesse Yates
240-888-2200
@jesse_yates