Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Question on Coprocessors and Atomicity


Copy link to this message
-
Re: Question on Coprocessors and Atomicity
Suraj Varma 2011-12-04, 16:39
Jesse:
>> Quick soln - write a CP to check the single row (blocking the put).

Yeah - given that I want this to be atomically done, I'm wondering if
this would even work (because, I believe I'd need to unlock the row so
that the checkAndMutate can take the lock - so, there is a brief
window between where there is no lock being held - and some other
thread could take that lock). One option would be to pass in a lock to
checkAndMutate ... but that would increase the locking period and may
have performance implications, I think.

I see a lot of potential in the Constraints implementation - it would
really open up CAS operations to do functional constraint checking,
rather than just value comparisons.

--Suraj

On Sun, Dec 4, 2011 at 8:32 AM, Suraj Varma <[EMAIL PROTECTED]> wrote:
> Thanks - I see that the lock is taken internal to checkAndMutate.
>
> I'm wondering whether it is a better idea to actually pass in a
> Constraint (or even Constraints) as the checkAndMutate argument. Right
> now it is taking in an Comparator and a CompareOp for verification.
> But, this could just be a special case of Constraint which is
> evaluated within the lock.
>
> In other words, we could open up a richer Constraint checking api
> where any "functional" Constraint check can be performed in the
> checkAndPut operation.
>
> This would also not have the same performance impact of taking a
> rowLock in preCheckAndPut and release in postCheckAndPut. And - it is
> really (in my mind) implementing the compare-and-set more generically.
>
> I also see the potential of passing in multiple constraints (say
> upper/lower bounds in Increment/Decrement operations) etc.
>
> --Suraj
>
>
> On Sat, Dec 3, 2011 at 7:44 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> From HRegionServer.checkAndPut():
>>    if (region.getCoprocessorHost() != null) {
>>      Boolean result = region.getCoprocessorHost()
>>        .preCheckAndPut(row, family, qualifier, CompareOp.EQUAL, comparator,
>>          put);
>> ...
>>    boolean result = checkAndMutate(regionName, row, family, qualifier,
>>      CompareOp.EQUAL, new BinaryComparator(value), put,
>>      lock);
>> We can see that the lock isn't taken for preCheckAndPut().
>>
>> To satisfy Suraj's requirement, I think a slight change to checkAndPut() is
>> needed so that atomicity can be achieved across preCheckAndPut() and
>> checkAndMutate().
>>
>> Cheers
>>
>> On Sat, Dec 3, 2011 at 4:54 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>>
>>> Just so my question is clear ... everything I'm suggesting is in the
>>> context of a single row (not cross row / table). - so, yes, I'm
>>> guessing obtaining a RowLock on the region side during preCheckAndPut
>>> / postCheckAndPut would certainly work. Which was why I was asking
>>> whether the pre/postCheckAndPut obtains the row lock or whether the
>>> row lock is only obtained within checkAndPut.
>>>
>>> Let's say the coprocessor takes a rowlock in preCheckAndPut ... will
>>> that even work? i.e. can the same rowlock be inherited by the
>>> checkAndPut api within that thread's context? Or will preCheckAndPut
>>> have to release the lock so that checkAndPut can take it (which won't
>>> work for my case, as it has to be atomic between the preCheck and
>>> Put.)
>>>
>>> Thanks for pointing me to the Constraints functionality - I'll take a
>>> look at whether it could potentially work.
>>> --Suraj
>>>
>>> On Sat, Dec 3, 2011 at 10:25 AM, Jesse Yates <[EMAIL PROTECTED]>
>>> wrote:
>>> > I think the feature you are looking for is a Constraint. Currently they
>>> are
>>> > being added to 0.94 in
>>> > HBASE-4605<https://issues.apache.org/jira/browse/HBASE-4605>;
>>> > they are almost ready to be rolled in, and backporting to 0.92 is
>>> > definitely doable.
>>> >
>>> > However, Constraints aren't going to be quite flexible enough to
>>> > efficiently support what you are describing. For instance, with a
>>> > constraint, you are ideally just checking the put value against some
>>> simple
>>>