Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Re: Best technique for doing lookup with Secondary Index


+
anil gupta 2012-10-26, 15:14
+
anil gupta 2012-10-26, 16:43
+
fding hbase 2012-10-26, 10:14
+
Jerry Lam 2012-10-26, 14:29
Copy link to this message
-
RE: Best technique for doing lookup with Secondary Index
Ramkrishna.S.Vasudevan 2012-10-26, 14:33
Yes we can do this, but for it to happen you may have to have your custom
load balancer which will help you in getting the collocation.

Regards
Ram

> -----Original Message-----
> From: Jerry Lam [mailto:[EMAIL PROTECTED]]
> Sent: Friday, October 26, 2012 7:59 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Best technique for doing lookup with Secondary Index
>
> Can we enforce 2 regions to collocate together as a logical group?
>
> On Fri, Oct 26, 2012 at 6:14 AM, fding hbase <[EMAIL PROTECTED]>
> wrote:
>
> > https://github.com/danix800/hbase-indexed
> >
> > On Fri, Oct 26, 2012 at 4:13 PM, Ramkrishna.S.Vasudevan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > > AFAIK, RPC cannot be avoided even if Region A and Region B are on
> same
> > > > RS
> > > > since these two regions are from different table. Am i right?
> > >
> > > No... suppose your Region A and Region B of different tables are
> > collocated
> > > on same RS then from the coprocessor environment variable you can
> get
> > > access
> > > to the RS.
> > > From RS you can get the online regions and from that region object
> you
> > can
> > > call puts or gets.  This will not involve any RPC with in that RS
> because
> > > we
> > > only deal with Region objects.
> > >
> > > Regards
> > > Ram
> > >
> > > > -----Original Message-----
> > > > From: anil gupta [mailto:[EMAIL PROTECTED]]
> > > > Sent: Friday, October 26, 2012 12:17 PM
> > > > To: [EMAIL PROTECTED]
> > > > Subject: Re: Best technique for doing lookup with Secondary Index
> > > >
> > > > >
> > > > > Now your main question is lookups right
> > > > > Now there are some more hooks in the scan flow called
> > > > pre/postScannerOpen,
> > > > > pre/postScannerNext.
> > > > > May be you can try using them to do a look up on the secondary
> table
> > > > and
> > > > > then use those values and pass it to the main table next().
> > > > >
> > > >
> > > > In secondary index its hard to avoid at-least two RPC calls(1
> from
> > > > client
> > > > to table B and then from table B to Table A) whether you use
> coproc or
> > > > not.
> > > > But, i believe using coproc is better than doing RPC calls from
> client
> > > > since it might be outside the subnet/network of cluster. In this
> case,
> > > > the
> > > > RPC will be faster when we use coprocs. In my case the client is
> > > > certainly
> > > > not in the same subnet or network zone. I need to provide results
> of
> > > > query
> > > > in around 100 milliseconds or less so i need to be really frugal.
> Let
> > > > me
> > > > know your views on this.
> > > >
> > > > Have you implemented queries with Secondary indexes using coproc
> yet?
> > > > At present i have tried the client side query and i can get the
> results
> > > > of
> > > > query in around 100 ms. I am enticed to try out the coproc
> > > > implementation.
> > > >
> > > > But this may involve more RPC calls as your regions of "A" and
> "B" may
> > > > be in
> > > > > different RS.
> > > > >
> > > > AFAIK, RPC cannot be avoided even if Region A and Region B are on
> same
> > > > RS
> > > > since these two regions are from different table. Am i right?
> > > >
> > > >
> > > > Thanks,
> > > > Anil Gupta
> > > >
> > > > On Thu, Oct 25, 2012 at 9:20 PM, Ramkrishna.S.Vasudevan <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > > > > Is it a
> > > > > > good idea to create Htable instance on "B" and do put in my
> mapper?
> > > > I
> > > > > > might
> > > > > > try this idea.
> > > > > Yes you can do this..  May be the same mapper you can do a put
> for
> > > > table
> > > > > "B".  This was how we have tried loading data to another table
> by
> > > > using the
> > > > > main table "A"
> > > > > Puts.
> > > > >
> > > > > Now your main question is lookups right
> > > > > Now there are some more hooks in the scan flow called
> > > > pre/postScannerOpen,
> > > > > pre/postScannerNext.
> > > > > May be you can try using them to do a look up on the secondary
> table
> > > > and
> > > > > then use those values and pass it to the main table next().
+
anil gupta 2012-10-26, 06:46
+
Ramkrishna.S.Vasudevan 2012-10-26, 08:13
+
anil gupta 2012-10-24, 21:40
+
Anoop Sam John 2012-10-25, 04:44
+
Ramkrishna.S.Vasudevan 2012-10-25, 05:16
+
anil gupta 2012-10-25, 22:10
+
Ramkrishna.S.Vasudevan 2012-10-26, 04:20
+
Anoop Sam John 2012-10-26, 04:33
+
Anoop Sam John 2012-10-26, 04:06
+
anil gupta 2012-10-26, 04:44
+
Doug Meil 2012-10-27, 00:35