Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Question on Coprocessors and Atomicity


Copy link to this message
-
Re: Question on Coprocessors and Atomicity
Hi:
I opened a jira ticket on this: https://issues.apache.org/jira/browse/HBASE-4999

I have linked to HBASE-4605 in the description to show related work on
Constraints by Jesse.

Thanks!
--Suraj

On Sun, Dec 4, 2011 at 1:10 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> Currently ConstraintProcessor latches onto prePut() to perform validation
> check.
>
> From HRegion.doMiniBatchPut() where prePut() is called:
>    /* Run coprocessor pre hook outside of locks to avoid deadlock */
> So to make use of Constraint in Suraj's scenario, we have some decisions to
> make about various factors.
>
> Cheers
>
> On Sun, Dec 4, 2011 at 8:39 AM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>
>> Jesse:
>> >> Quick soln - write a CP to check the single row (blocking the put).
>>
>> Yeah - given that I want this to be atomically done, I'm wondering if
>> this would even work (because, I believe I'd need to unlock the row so
>> that the checkAndMutate can take the lock - so, there is a brief
>> window between where there is no lock being held - and some other
>> thread could take that lock). One option would be to pass in a lock to
>> checkAndMutate ... but that would increase the locking period and may
>> have performance implications, I think.
>>
>> I see a lot of potential in the Constraints implementation - it would
>> really open up CAS operations to do functional constraint checking,
>> rather than just value comparisons.
>>
>> --Suraj
>>
>> On Sun, Dec 4, 2011 at 8:32 AM, Suraj Varma <[EMAIL PROTECTED]> wrote:
>> > Thanks - I see that the lock is taken internal to checkAndMutate.
>> >
>> > I'm wondering whether it is a better idea to actually pass in a
>> > Constraint (or even Constraints) as the checkAndMutate argument. Right
>> > now it is taking in an Comparator and a CompareOp for verification.
>> > But, this could just be a special case of Constraint which is
>> > evaluated within the lock.
>> >
>> > In other words, we could open up a richer Constraint checking api
>> > where any "functional" Constraint check can be performed in the
>> > checkAndPut operation.
>> >
>> > This would also not have the same performance impact of taking a
>> > rowLock in preCheckAndPut and release in postCheckAndPut. And - it is
>> > really (in my mind) implementing the compare-and-set more generically.
>> >
>> > I also see the potential of passing in multiple constraints (say
>> > upper/lower bounds in Increment/Decrement operations) etc.
>> >
>> > --Suraj
>> >
>> >
>> > On Sat, Dec 3, 2011 at 7:44 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> >> From HRegionServer.checkAndPut():
>> >>    if (region.getCoprocessorHost() != null) {
>> >>      Boolean result = region.getCoprocessorHost()
>> >>        .preCheckAndPut(row, family, qualifier, CompareOp.EQUAL,
>> comparator,
>> >>          put);
>> >> ...
>> >>    boolean result = checkAndMutate(regionName, row, family, qualifier,
>> >>      CompareOp.EQUAL, new BinaryComparator(value), put,
>> >>      lock);
>> >> We can see that the lock isn't taken for preCheckAndPut().
>> >>
>> >> To satisfy Suraj's requirement, I think a slight change to
>> checkAndPut() is
>> >> needed so that atomicity can be achieved across preCheckAndPut() and
>> >> checkAndMutate().
>> >>
>> >> Cheers
>> >>
>> >> On Sat, Dec 3, 2011 at 4:54 PM, Suraj Varma <[EMAIL PROTECTED]>
>> wrote:
>> >>
>> >>> Just so my question is clear ... everything I'm suggesting is in the
>> >>> context of a single row (not cross row / table). - so, yes, I'm
>> >>> guessing obtaining a RowLock on the region side during preCheckAndPut
>> >>> / postCheckAndPut would certainly work. Which was why I was asking
>> >>> whether the pre/postCheckAndPut obtains the row lock or whether the
>> >>> row lock is only obtained within checkAndPut.
>> >>>
>> >>> Let's say the coprocessor takes a rowlock in preCheckAndPut ... will
>> >>> that even work? i.e. can the same rowlock be inherited by the
>> >>> checkAndPut api within that thread's context? Or will preCheckAndPut
>> >>> have to release the lock so that checkAndPut can take it (which won't
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB