Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - HBase - Secondary Index


+
Anoop Sam John 2012-12-04, 08:10
+
Jan Van Besien 2012-12-04, 19:24
+
Anoop Sam John 2012-12-05, 11:04
+
ramkrishna vasudevan 2012-12-05, 11:28
+
ramkrishna vasudevan 2012-12-05, 11:40
+
Jan Van Besien 2012-12-05, 15:12
+
ramkrishna vasudevan 2012-12-05, 17:42
+
Andrey Stepachev 2012-12-05, 11:52
+
Anoop Sam John 2012-12-05, 12:55
+
Andrey Stepachev 2012-12-05, 15:59
+
Anoop John 2012-12-05, 17:54
+
Jonathan Hsieh 2012-12-05, 19:23
+
lars hofhansl 2012-12-05, 21:03
+
Andrew Purtell 2012-12-06, 04:41
+
Anoop Sam John 2012-12-06, 04:57
Copy link to this message
-
Re: HBase - Secondary Index
ramkrishna vasudevan 2012-12-06, 13:05
Yes there is a one to one mapping, but even if 5 indices are there still we
will have only all the indices data in that same region.

May be more perf testing with more regions is needed.

Anoop, need to tell something on the range scans?

Regards
Ram

On Thu, Dec 6, 2012 at 10:27 AM, Anoop Sam John <[EMAIL PROTECTED]> wrote:

> >I'd guess that would look like a coprocessor based solution, so
> there's no cost to those who don't want to use it, and minimal changes
> to core code.
>
> Yes Andrew, This was the main consideration from our side when we were
> doing the design of the secondary indexing solution here.  Plus the minimum
> possible degradation for the Put as our customer was very much on to it :)
>
> >Everyone asks for it. All the time.
> Yes this is what we also been hearing from our customers. :)
>
> -Anoop-
> ________________________________________
> From: Andrew Purtell [[EMAIL PROTECTED]]
> Sent: Thursday, December 06, 2012 10:11 AM
> To: [EMAIL PROTECTED]
> Subject: Re: HBase - Secondary Index
>
> We won't find a secondary indexing scheme for HBase that will fit all
> use cases.
>
> However I am becoming convinced we should distribute as part of the
> distribution something good enough for some subset of simple/common
> use cases, so users have *something* to try, and can play around it.
> Everyone asks for it. All the time.
>
> I'd guess that would look like a coprocessor based solution, so
> there's no cost to those who don't want to use it, and minimal changes
> to core code.
>
> Maybe it would suit them, or if not at least they will have enough
> experience having played around with it to decide why they might need
> to do something else and what that might look like for their use case.
>
> On 12/6/12, lars hofhansl <[EMAIL PROTECTED]> wrote:
> > I'd say "it depends".
> > It seems everybody wants secondary indexes in HBase. The problem is that
> > most folks don't agree what that actually means.
> >
> > The most interesting problem and discussion point (IMHO) is that HBase
> would
> > need some form of schema description.
> >
> > See also HBASE-7221. Assuming we had something like that, it could be a
> > building block for a simple built-in secondary index solution.
> >
> > In the end you're probably right and we cannot prescribe a single
> secondary
> > index solution that will fit all use cases.
> >
> > -- Lars
> >
> >
> > ----- Original Message -----
> > From: Jonathan Hsieh <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Cc:
> > Sent: Wednesday, December 5, 2012 11:23 AM
> > Subject: Re: HBase - Secondary Index
> >
> > I personally feel that we should only add the primitive apis necessary to
> > make this work (and if non are required, all the better!), and to try to
> > keep secondary indexing work it as a separate project on top of hbase
> > because there are many possible valid architectures and  implementations.
> >
> > I have two question areas -- one touched upon in previous messages --
> what
> > logging and how do we get consistency guarantee's do we get bewteen the
> > index and primary?
> >
> > The other has to do with scalability. I'm not sure I interpreted the
> slides
> > correctly, but from the slides 8 and 13, is the architecture such that
> each
> > primary table region has a corresponding index table region?
> >
> > Is slide 14 a comparison of a full table scan vs the indexed lookups on 4
> > rs's?  What happens if we go up to 20, or 100 rs's?  If I'm right about
> the
> > per index region per table region, I have a feeling this isn't going to
> > scale well with a large number of regions (since it would potentially
> have
> > to talk to each region and essentially every region server).
> >
> > Jon.
> >
> > On Wed, Dec 5, 2012 at 9:54 AM, Anoop John <[EMAIL PROTECTED]>
> wrote:
> >
> >> I mean HBase devs to work on having sec indexing available with the
> HBase
> >> distribution...  Now I guess many users of HBase implement different
> kinds
> >> of sec indexing ..:)
> >
+
Doug Meil 2012-12-05, 21:58
+
Anoop John 2012-12-06, 01:17
+
Nick Dimiduk 2012-12-18, 17:48
+
Andrew Purtell 2012-12-19, 00:51
+
ramkrishna vasudevan 2012-12-19, 04:24