Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Design review: Secondary index support through coprocessors


Copy link to this message
-
Re: Design review: Secondary index support through coprocess
Wow.

That's the first time in 25 years that I've heard someone actually reference the dining philosophers problem.
;-)

On Jan 22, 2014, at 1:35 PM, Wei Tan <[EMAIL PROTECTED]> wrote:

> Thanks, Vladimir. So a RPC call RS1 --> RS2 takes two handlers, one from
> RS1 and one from RS2? If that is true, then I understand that it is a
> typical Dining philosophers problem.
>
> Maybe a random yielding mechanism can solve this problem.
> Best regards,
> Wei
>
> ---------------------------------
> Wei Tan, PhD
> Research Staff Member
> IBM T. J. Watson Research Center
> http://researcher.ibm.com/person/us-wtan
>
>
>
> From:   Vladimir Rodionov <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   01/22/2014 12:09 PM
> Subject:        RE: Design review: Secondary index support through
> coprocess
>
>
>
> Deadlocks are possible because  cross region RPCs create cyclic
> dependencies in HBase cluster.
>
> RS1-> RS2->RS3->RS1, where -> is PRC call
>
> now imagine that last call from RS3 to RS1 is blocked because there no
> more available handler threads to process it.
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: Wei Tan [[EMAIL PROTECTED]]
> Sent: Wednesday, January 22, 2014 7:51 AM
> To: [EMAIL PROTECTED]
> Subject: RE: Design review: Secondary index support through coprocess
>
> Why cross-RS RPC is going to cause deadlocks? It is a matter of logic
> incorrectness, or resource outage? Say, if we set the #handler to be
> large, logically deadlock still occurs?
> Best regards,
> Wei
>
>
>
>
> From:   Vladimir Rodionov <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   01/20/2014 03:00 PM
> Subject:        RE: Design review: Secondary index support through
> coprocess
>
>
>
>>> Yes, the coprocessors potentially cross RS boundaries.
>
> The open path to the disaster. Inter region RPCs in coprocessors may
> result in periodic cluster - wide deadlocks
>
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: James Taylor [[EMAIL PROTECTED]]
> Sent: Monday, January 20, 2014 11:39 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Design review: Secondary index support through coprocess
>
> Yes, the coprocessors potentially cross RS boundaries. No, the index is
> not
> co-located with the main table. Take a look at the link I sent as that
> should be able to answer a lot of questions.
>
> Thanks,
> James
>
>
> On Mon, Jan 20, 2014 at 11:03 AM, Michael Segel
> <[EMAIL PROTECTED]>wrote:
>
>> James,
>>
>> Ok…
>>
>> Its been a while since we talked about this…
>>
>> While the index is in a separate table, is that table being split and
>> collocated with the main table?
>>
>> If you’re using the coprocessor to maintain the index, that would imply
>> you’re crossing RS boundaries if your index is truly orthogonal.
>>
>> Is this what you’re doing?
>>
>> On Jan 20, 2014, at 11:32 AM, James Taylor <[EMAIL PROTECTED]>
> wrote:
>>
>>> Mike,
>>> Yes, you're mistaken:
>>> - secondary indexes in Phoenix are orthogonal to the base table.
> They're
>> in
>>> a separate table (
>>> http://phoenix.incubator.apache.org/secondary_indexing.html).
>>> - Phoenix has joins. They're in our master branch with a release
>> scheduled
>>> for next month
>>> - numeric strings? Not a use case for indexing numeric data? Have you
>> ever
>>> seen a number used as an ID?
>>> Thanks,
>>> James
>>>
>>>
>>> On Mon, Jan 20, 2014 at 8:50 AM, Michael Segel <
>> [EMAIL PROTECTED]>wrote:
>>>
>>>> Indexes tend to be orthogonal to the base table, not to mention if
>> you’re
>>>> using an inverted table for an index, your index table would be much
>>>> thinner than your base table.

The opinions expressed here are mine, while they may reflect a cognitive thought, that is purely accidental.
Use at your own risk.
Michael Segel
michael_segel (AT) hotmail.com
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB