Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Coprocessor Increments


Copy link to this message
-
Re: Coprocessor Increments
John,

How are you creating your htable connection?
Assuming that the RO is overloading the post_put() method, are you instantiating a new HTable connection each time or are you creating a pool at instantiation and then getting a connection from the pool?

It may be that you're not seeing a deadlock issue in the thread handlers.

Also how fast are you adding rows?

I'm wondering if you're running in to some thread safe issues...
On Oct 11, 2013, at 8:06 PM, John Weatherford <[EMAIL PROTECTED]> wrote:

> OP Here :)
>
> Our current design involves a Region Observer on a table that does increments on a second table. We took the approach that Michael said and inside the RO, we got a new connection and everything. We believe this is causing deadlocks for us. Our next attempt is going to be writing to another row in the same table where we will store the increments. If this doesn't work, we are going to simply pull the increments out of the RO and do them in the application or in Flume.
>
> @Tom Brown
>  I would be very interested to hear more about your solution of aggregating the increments in another system that is then responsible for updating in Hbase.
>
> -jW
>
> On Fri 11 Oct 2013 10:26:58 AM PDT, Vladimir Rodionov wrote:
>>>> With respect to the OP's design… does the deadlock occur because he's trying to update a column in a different row within the same table?
>>
>> Because he is trying to update *row* in a different Region (and potentially in different RS).
>>
>> Best regards,
>> Vladimir Rodionov
>> Principal Platform Engineer
>> Carrier IQ, www.carrieriq.com
>> e-mail: [EMAIL PROTECTED]
>>
>> ________________________________________
>> From: Michael Segel [[EMAIL PROTECTED]]
>> Sent: Friday, October 11, 2013 9:10 AM
>> To: [EMAIL PROTECTED]
>> Cc: Vladimir Rodionov
>> Subject: Re: Coprocessor Increments
>>
>>
>> Confidentiality Notice:  The information contained in this message, including any attachments hereto, may be confidential and is intended to be read only by the individual or entity to whom this message is addressed. If the reader of this message is not the intended recipient or an agent or designee of the intended recipient, please note that any review, use, disclosure or distribution of this message or its attachments, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or [EMAIL PROTECTED] and delete or destroy any copy of this message and its attachments.
>

The opinions expressed here are mine, while they may reflect a cognitive thought, that is purely accidental.
Use at your own risk.
Michael Segel
michael_segel (AT) hotmail.com