Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Re: HBase - Secondary Index


Copy link to this message
-
Re: HBase - Secondary Index
ramkrishna vasudevan 2012-12-14, 11:34
Nice explanation Anoop.  :)
No prefix filters will be needed to query the secondary index table.  As
Anoop told
<Startkey><IndexName>-> Static part
<Value>-> Main table rowkey value
<Actualrowkey>Actual rowkey.

So you just need to set a start row with <StartKey><IndexName><Value>...
This will give you the Actual rowkey as it is part of the rowkey..
Just use this rowkey on the primary table...You get the exact row needed...
All are server side...Nothing comes to the client till the final actual row
key is fetched..

Regards
Ram

On Fri, Dec 14, 2012 at 2:24 PM, Anoop Sam John <[EMAIL PROTECTED]> wrote:

> Hi Anil,
>
> >1. In your presentation you mentioned that region of Primary Table and
> Region of Secondary Table are always located on the same region server. How
> do you achieve it? By using the Primary table rowkey as prefix of  Rowkey
> of Secondary Table? Will your implementation work if the rowkey of primary
> table cannot be used as prefix in rowkey of Secondary table( i have this
> limitation in my use case)?
> First all there will be same number of regions in both primary and index
> tables. All the start/stop keys of the regions also will be same.
> Suppose there are 2 regions on main table say for keys 0-10 and 10-20.
>  Then we will create 2 regions in index table also with same key ranges.
> At the master balancing level it is easy to collocate these regions seeing
> the start and end keys.
> When the selection of the rowkey that will be used in the index table is
> the key here.
> What we will do is all the rowkeys in the index table will be prefixed
> with the start key of the region/
> When an entry is added to the main table with rowkey as 5 it will go to
> the 1st region (0-10)
> Now there will be index region with range as 0-10.  We will select this
> region to store this index data.
> The row getting added into the index region for this entry will have a
> rowkey 0_x_5
> I am just using '_' as a seperator here just to show this. Actually we
> wont be having any seperator.
> So the rowkeys (in index region) will have a static begin part always.
>  Will scan time also we know this part and so the startrow and endrow
> creation for the scan will be possible.. Note that we will store the actual
> table row key as the last part of the index rowkey itself not as a value.
> This is better option in our case of handling the scan index usage also at
> sever side.  There is no index data fetch to client side..
>
> I feel your use case perfectly fit with our model
>
> >2. Are you using an Endpoint or Observer for building the secondary index
> table?
> Observer
>
> >3. "Custom balancer do collocation". Is it a custom load balancer of HBase
> Master or something else?
> It is a balancer implementation which will be plugged into Master
>
> >4. Your region split looks interesting. I dont have much info about it.
> Can
> you point to some docs on IndexHalfStoreFileReader?
> Sorry I am not able to publish any design doc or code as the company has
> not decided to open src the solution yet.
> Any particular query you come acorss pls feel free to aske me :)
> You can see the HalfStoreFileReader class 1st..
>
> -Anoop-
> ________________________________________
> From: anil gupta [[EMAIL PROTECTED]]
> Sent: Friday, December 14, 2012 2:11 PM
> To: [EMAIL PROTECTED]
> Subject: Re: HBase - Secondary Index
>
> Hi Anoop,
>
> Nice presentation and seems like a smart implementation. Since the
> presentation only covered bullet points so i have couple of questions on
> your implementation. :)
>
> Here is a recap to my implementation and our previous discussion on
> Secondary index:
>
> Here is the link to previous email thread:
> http://search-hadoop.com/m/1zWPMaaRtr .
>
> The secondary index is stored in table "B" as rowkey B --> family:<rowkey
> A>  . "<rowkey A>" is the column qualifier. Every row in B will only on
> have one column "k" and the value of that column is the rowkey of A.
>
> Suppose i am storing customer events in table A. I have two requirement for