Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> what's the roadmap of secondary index of hbase?


Copy link to this message
-
Re: what's the roadmap of secondary index of hbase?
On 2/25/11 12:43 PM, Jonathan Gray wrote:
> Cool.  Plans for a design phase that we can collaborate on?
>
Hi Jon,

I'm thinking that we could use a coprocessor that watches the
Write-Ahead Log (using the WAL-edit operations  
https://issues.apache.org/jira/browse/HBASE-3257 "Coprocessors: Extend
server side integration API to include HLog operations"). This
coprocessor would write these edits, perhaps filtering or transforming
them, and enqueing the results in a global queue. A separate process
would be responsible for pulling operations off the queue and using
HBase client operations to do the insert into a secondary index table
appropriate for that operation.
     Perhaps we could use some of the work that the Lily people have
done with HBase indexing (see
http://www.lilyproject.org/lily/about/playground/hbaseindexes.html) in
order to do the edit->hbase operation transformations and the secondary
index table creation.
     -Eugene