Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Design review: Secondary index support through coprocessors


Copy link to this message
-
Re: Design review: Secondary index support through coprocess
Wow.

That's the first time in 25 years that I've heard someone actually reference the dining philosophers problem.
;-)

On Jan 22, 2014, at 1:35 PM, Wei Tan <[EMAIL PROTECTED]> wrote:

> Thanks, Vladimir. So a RPC call RS1 --> RS2 takes two handlers, one from
> RS1 and one from RS2? If that is true, then I understand that it is a
> typical Dining philosophers problem.
>
> Maybe a random yielding mechanism can solve this problem.
> Best regards,
> Wei
>
> ---------------------------------
> Wei Tan, PhD
> Research Staff Member
> IBM T. J. Watson Research Center
> http://researcher.ibm.com/person/us-wtan
>
>
>
> From:   Vladimir Rodionov <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   01/22/2014 12:09 PM
> Subject:        RE: Design review: Secondary index support through
> coprocess
>
>
>
> Deadlocks are possible because  cross region RPCs create cyclic
> dependencies in HBase cluster.
>
> RS1-> RS2->RS3->RS1, where -> is PRC call
>
> now imagine that last call from RS3 to RS1 is blocked because there no
> more available handler threads to process it.
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: Wei Tan [[EMAIL PROTECTED]]
> Sent: Wednesday, January 22, 2014 7:51 AM
> To: [EMAIL PROTECTED]
> Subject: RE: Design review: Secondary index support through coprocess
>
> Why cross-RS RPC is going to cause deadlocks? It is a matter of logic
> incorrectness, or resource outage? Say, if we set the #handler to be
> large, logically deadlock still occurs?
> Best regards,
> Wei
>
>
>
>
> From:   Vladimir Rodionov <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   01/20/2014 03:00 PM
> Subject:        RE: Design review: Secondary index support through
> coprocess
>
>
>
>>> Yes, the coprocessors potentially cross RS boundaries.
>
> The open path to the disaster. Inter region RPCs in coprocessors may
> result in periodic cluster - wide deadlocks
>
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: James Taylor [[EMAIL PROTECTED]]
> Sent: Monday, January 20, 2014 11:39 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Design review: Secondary index support through coprocess
>
> Yes, the coprocessors potentially cross RS boundaries. No, the index is
> not
> co-located with the main table. Take a look at the link I sent as that
> should be able to answer a lot of questions.
>
> Thanks,
> James
>
>
> On Mon, Jan 20, 2014 at 11:03 AM, Michael Segel
> <[EMAIL PROTECTED]>wrote:
>
>> James,
>>
>> Ok…
>>
>> Its been a while since we talked about this…
>>
>> While the index is in a separate table, is that table being split and
>> collocated with the main table?
>>
>> If you’re using the coprocessor to maintain the index, that would imply
>> you’re crossing RS boundaries if your index is truly orthogonal.
>>
>> Is this what you’re doing?
>>
>> On Jan 20, 2014, at 11:32 AM, James Taylor <[EMAIL PROTECTED]>
> wrote:
>>
>>> Mike,
>>> Yes, you're mistaken:
>>> - secondary indexes in Phoenix are orthogonal to the base table.
> They're
>> in
>>> a separate table (
>>> http://phoenix.incubator.apache.org/secondary_indexing.html).
>>> - Phoenix has joins. They're in our master branch with a release
>> scheduled
>>> for next month
>>> - numeric strings? Not a use case for indexing numeric data? Have you
>> ever
>>> seen a number used as an ID?
>>> Thanks,
>>> James
>>>
>>>
>>> On Mon, Jan 20, 2014 at 8:50 AM, Michael Segel <
>> [EMAIL PROTECTED]>wrote:
>>>
>>>> Indexes tend to be orthogonal to the base table, not to mention if
>> you’re
>>>> using an inverted table for an index, your index table would be much
>>>> thinner than your base table.

The opinions expressed here are mine, while they may reflect a cognitive thought, that is purely accidental.
Use at your own risk.
Michael Segel
michael_segel (AT) hotmail.com