Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Design review: Secondary index support through coprocessors


+
Ted Yu 2014-01-08, 20:41
+
Sudarshan Kadambi 2014-01-09, 13:58
+
rajesh babu Chintaguntla 2014-01-09, 15:59
+
Anoop John 2014-01-09, 16:20
+
Ted Yu 2014-01-09, 17:09
+
Jesse Yates 2014-01-09, 14:36
+
James Taylor 2014-01-09, 16:48
+
Michael Segel 2014-01-20, 16:50
+
James Taylor 2014-01-20, 17:32
+
Michael Segel 2014-01-20, 19:03
+
James Taylor 2014-01-20, 19:39
+
Ted Yu 2014-01-20, 19:42
+
Michael Segel 2014-01-20, 19:53
+
Vladimir Rodionov 2014-01-20, 19:57
+
Michael Segel 2014-01-20, 22:00
+
lars hofhansl 2014-01-20, 21:54
+
Andrew Purtell 2014-01-20, 22:45
+
Jesse Yates 2014-01-20, 22:53
+
Andrew Purtell 2014-01-20, 23:03
+
ramkrishna vasudevan 2014-01-21, 04:14
+
Michael Segel 2014-01-21, 12:34
+
Wei Tan 2014-01-22, 15:51
+
Vladimir Rodionov 2014-01-22, 17:06
+
Wei Tan 2014-01-22, 19:35
+
James Taylor 2014-01-22, 19:00
+
Wei Tan 2014-01-22, 19:38
Copy link to this message
-
RE: Design review: Secondary index support through coprocess
Because each server has regions from many tables, the handlers on server A
may all be busy trying to write to server B. However, server Bs handlers
may all be busy handling requests from the client to update server As
index... Deadlock
On Jan 22, 2014 7:52 AM, "Wei Tan" <[EMAIL PROTECTED]> wrote:

> Why cross-RS RPC is going to cause deadlocks? It is a matter of logic
> incorrectness, or resource outage? Say, if we set the #handler to be
> large, logically deadlock still occurs?
> Best regards,
> Wei
>
>
>
>
> From:   Vladimir Rodionov <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   01/20/2014 03:00 PM
> Subject:        RE: Design review: Secondary index support through
> coprocess
>
>
>
> >>Yes, the coprocessors potentially cross RS boundaries.
>
> The open path to the disaster. Inter region RPCs in coprocessors may
> result in periodic cluster - wide deadlocks
>
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: James Taylor [[EMAIL PROTECTED]]
> Sent: Monday, January 20, 2014 11:39 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Design review: Secondary index support through coprocess
>
> Yes, the coprocessors potentially cross RS boundaries. No, the index is
> not
> co-located with the main table. Take a look at the link I sent as that
> should be able to answer a lot of questions.
>
> Thanks,
> James
>
>
> On Mon, Jan 20, 2014 at 11:03 AM, Michael Segel
> <[EMAIL PROTECTED]>wrote:
>
> > James,
> >
> > Ok…
> >
> > Its been a while since we talked about this…
> >
> > While the index is in a separate table, is that table being split and
> > collocated with the main table?
> >
> > If you’re using the coprocessor to maintain the index, that would imply
> > you’re crossing RS boundaries if your index is truly orthogonal.
> >
> > Is this what you’re doing?
> >
> > On Jan 20, 2014, at 11:32 AM, James Taylor <[EMAIL PROTECTED]>
> wrote:
> >
> > > Mike,
> > > Yes, you're mistaken:
> > > - secondary indexes in Phoenix are orthogonal to the base table.
> They're
> > in
> > > a separate table (
> > > http://phoenix.incubator.apache.org/secondary_indexing.html).
> > > - Phoenix has joins. They're in our master branch with a release
> > scheduled
> > > for next month
> > > - numeric strings? Not a use case for indexing numeric data? Have you
> > ever
> > > seen a number used as an ID?
> > > Thanks,
> > > James
> > >
> > >
> > > On Mon, Jan 20, 2014 at 8:50 AM, Michael Segel <
> > [EMAIL PROTECTED]>wrote:
> > >
> > >> Indexes tend to be orthogonal to the base table, not to mention if
> > you’re
> > >> using an inverted table for an index, your index table would be much
> > >> thinner than your base table.
> > >>
> > >> Having said that, the solution proposed by Yu, Taylor and others only
> > >> works if you want to use the index to help on server side filtering
> and
> > >> misses the boat on the larger and broader picture of improving query
> > >> optimization and joins.
> > >>
> > >> HINT: Unless I am mistaken… until you treat the index as orthogonal
> to
> > the
> > >> base table, you will always lag performance of traditional MPP DWs
> like
> > >> Informix XPS. (Now part of IBM’s IM pillar )
> > >>
> > >> In addition, until you fix coprocessors in general, you will have
> > >> scalability and performance issues.
> > >> (Note that you can write a coprocessor to create a sandbox and
> separate
> > >> the co-process from the RS jvm, however it would be better if it were
> > part
> > >> of the underlying coprocessor code. )
> > >>
> > >> The current implementation makes joins worthless.
> > >> (Note that in prior discussions,  Phoenix doesn’t do joins…)
> > >> Here’s why:
> > >> In order to do a join, if you use the proposed index, you have to
> first
> > >> reduce each index in to a single, sort ordered set.  Then you can
> take
> > the
> > >> intersection of the index result sets.  The final set would be in
+
Stack 2014-01-09, 17:15
+
rajeshbabu chintaguntla 2014-01-20, 11:08
+
Michael Segel 2014-01-23, 10:46