Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Coprocessor Increments


+
John Weatherford 2013-10-10, 01:43
+
Vladimir Rodionov 2013-10-10, 02:28
+
Ted Yu 2013-10-10, 02:39
+
Ted Yu 2013-10-10, 02:43
+
John Weatherford 2013-10-10, 03:26
+
Ted Yu 2013-10-10, 03:43
+
John Weatherford 2013-10-10, 04:03
+
Michael Segel 2013-10-10, 14:57
+
John Weatherford 2013-10-10, 16:15
+
Michael Segel 2013-10-10, 16:55
+
Vladimir Rodionov 2013-10-10, 19:09
+
Vladimir Rodionov 2013-10-10, 19:23
+
Tom Brown 2013-10-10, 20:20
+
Michael Segel 2013-10-10, 21:57
+
Vladimir Rodionov 2013-10-10, 23:52
+
Michael Segel 2013-10-11, 16:10
+
Vladimir Rodionov 2013-10-11, 17:26
+
Michael Segel 2013-10-11, 19:09
+
John Weatherford 2013-10-12, 01:06
+
Ted Yu 2013-10-12, 14:42
+
Michael Segel 2013-10-12, 23:04
Copy link to this message
-
Re: Coprocessor Increments
John,

My 2 cents:
I tried implementing Secondary Index by using Region Observers on Put. It
works well under low load. But, under heavy load the RO could not keep up
with load cross region server writes.
Then, i decided not to use RO as per Andrew's explanation and  I moved all
the logic of building secondary index tables on my HBase Client . Since
then, the system has been running fine under heavy load.
IMO, if you will use RO and do cross RS read/write then perhaps this will
become your bottleneck in HBase.
Is it possible for you to avoid RO and control the writes/updates from
client side?

Thanks,
Anil Gupta
On Fri, Oct 11, 2013 at 6:06 PM, John Weatherford <
[EMAIL PROTECTED]> wrote:

> OP Here :)
>
> Our current design involves a Region Observer on a table that does
> increments on a second table. We took the approach that Michael said and
> inside the RO, we got a new connection and everything. We believe this is
> causing deadlocks for us. Our next attempt is going to be writing to
> another row in the same table where we will store the increments. If this
> doesn't work, we are going to simply pull the increments out of the RO and
> do them in the application or in Flume.
>
> @Tom Brown
>   I would be very interested to hear more about your solution of
> aggregating the increments in another system that is then responsible for
> updating in Hbase.
>
>  -jW
>
>
> On Fri 11 Oct 2013 10:26:58 AM PDT, Vladimir Rodionov wrote:
>
>> With respect to the OP's design… does the deadlock occur because he's
>>>> trying to update a column in a different row within the same table?
>>>>
>>>
>> Because he is trying to update *row* in a different Region (and
>> potentially in different RS).
>>
>> Best regards,
>> Vladimir Rodionov
>> Principal Platform Engineer
>> Carrier IQ, www.carrieriq.com
>> e-mail: [EMAIL PROTECTED]
>>
>> ______________________________**__________
>> From: Michael Segel [[EMAIL PROTECTED]]
>> Sent: Friday, October 11, 2013 9:10 AM
>> To: [EMAIL PROTECTED]
>> Cc: Vladimir Rodionov
>> Subject: Re: Coprocessor Increments
>>
>>
>> Confidentiality Notice:  The information contained in this message,
>> including any attachments hereto, may be confidential and is intended to be
>> read only by the individual or entity to whom this message is addressed. If
>> the reader of this message is not the intended recipient or an agent or
>> designee of the intended recipient, please note that any review, use,
>> disclosure or distribution of this message or its attachments, in any form,
>> is strictly prohibited.  If you have received this message in error, please
>> immediately notify the sender and/or [EMAIL PROTECTED] and
>> delete or destroy any copy of this message and its attachments.
>>
>
--
Thanks & Regards,
Anil Gupta
+
Michael Segel 2013-10-13, 13:02
+
anil gupta 2013-10-13, 15:15
+
Michael Segel 2013-10-14, 14:50
+
Tom Brown 2013-10-14, 16:36
+
John Weatherford 2013-10-14, 20:39
+
anil gupta 2013-10-14, 22:25
+
Ted Yu 2013-10-14, 22:34
+
anil gupta 2013-10-15, 03:57
+
Michael Segel 2013-10-15, 18:12
+
anil gupta 2013-10-15, 20:47
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB