Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Question on Coprocessors and Atomicity


Copy link to this message
-
Re: Question on Coprocessors and Atomicity
Hi:
I opened a jira ticket on this: https://issues.apache.org/jira/browse/HBASE-4999

I have linked to HBASE-4605 in the description to show related work on
Constraints by Jesse.

Thanks!
--Suraj

On Sun, Dec 4, 2011 at 1:10 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> Currently ConstraintProcessor latches onto prePut() to perform validation
> check.
>
> From HRegion.doMiniBatchPut() where prePut() is called:
>    /* Run coprocessor pre hook outside of locks to avoid deadlock */
> So to make use of Constraint in Suraj's scenario, we have some decisions to
> make about various factors.
>
> Cheers
>
> On Sun, Dec 4, 2011 at 8:39 AM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>
>> Jesse:
>> >> Quick soln - write a CP to check the single row (blocking the put).
>>
>> Yeah - given that I want this to be atomically done, I'm wondering if
>> this would even work (because, I believe I'd need to unlock the row so
>> that the checkAndMutate can take the lock - so, there is a brief
>> window between where there is no lock being held - and some other
>> thread could take that lock). One option would be to pass in a lock to
>> checkAndMutate ... but that would increase the locking period and may
>> have performance implications, I think.
>>
>> I see a lot of potential in the Constraints implementation - it would
>> really open up CAS operations to do functional constraint checking,
>> rather than just value comparisons.
>>
>> --Suraj
>>
>> On Sun, Dec 4, 2011 at 8:32 AM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>> > Thanks - I see that the lock is taken internal to checkAndMutate.
>> >
>> > I'm wondering whether it is a better idea to actually pass in a
>> > Constraint (or even Constraints) as the checkAndMutate argument. Right
>> > now it is taking in an Comparator and a CompareOp for verification.
>> > But, this could just be a special case of Constraint which is
>> > evaluated within the lock.
>> >
>> > In other words, we could open up a richer Constraint checking api
>> > where any "functional" Constraint check can be performed in the
>> > checkAndPut operation.
>> >
>> > This would also not have the same performance impact of taking a
>> > rowLock in preCheckAndPut and release in postCheckAndPut. And - it is
>> > really (in my mind) implementing the compare-and-set more generically.
>> >
>> > I also see the potential of passing in multiple constraints (say
>> > upper/lower bounds in Increment/Decrement operations) etc.
>> >
>> > --Suraj
>> >
>> >
>> > On Sat, Dec 3, 2011 at 7:44 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> >> From HRegionServer.checkAndPut():
>> >>    if (region.getCoprocessorHost() != null) {
>> >>      Boolean result = region.getCoprocessorHost()
>> >>        .preCheckAndPut(row, family, qualifier, CompareOp.EQUAL,
>> comparator,
>> >>          put);
>> >> ...
>> >>    boolean result = checkAndMutate(regionName, row, family, qualifier,
>> >>      CompareOp.EQUAL, new BinaryComparator(value), put,
>> >>      lock);
>> >> We can see that the lock isn't taken for preCheckAndPut().
>> >>
>> >> To satisfy Suraj's requirement, I think a slight change to
>> checkAndPut() is
>> >> needed so that atomicity can be achieved across preCheckAndPut() and
>> >> checkAndMutate().
>> >>
>> >> Cheers
>> >>
>> >> On Sat, Dec 3, 2011 at 4:54 PM, Suraj Varma <[EMAIL PROTECTED]>
>> wrote:
>> >>
>> >>> Just so my question is clear ... everything I'm suggesting is in the
>> >>> context of a single row (not cross row / table). - so, yes, I'm
>> >>> guessing obtaining a RowLock on the region side during preCheckAndPut
>> >>> / postCheckAndPut would certainly work. Which was why I was asking
>> >>> whether the pre/postCheckAndPut obtains the row lock or whether the
>> >>> row lock is only obtained within checkAndPut.
>> >>>
>> >>> Let's say the coprocessor takes a rowlock in preCheckAndPut ... will
>> >>> that even work? i.e. can the same rowlock be inherited by the
>> >>> checkAndPut api within that thread's context? Or will preCheckAndPut
>> >>> have to release the lock so that checkAndPut can take it (which won't