Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> what's the roadmap of secondary index of hbase?


Copy link to this message
-
Re: what's the roadmap of secondary index of hbase?
On 2/25/11 12:43 PM, Jonathan Gray wrote:
> Cool.  Plans for a design phase that we can collaborate on?
>
Hi Jon,

I'm thinking that we could use a coprocessor that watches the
Write-Ahead Log (using the WAL-edit operations  
https://issues.apache.org/jira/browse/HBASE-3257 "Coprocessors: Extend
server side integration API to include HLog operations"). This
coprocessor would write these edits, perhaps filtering or transforming
them, and enqueing the results in a global queue. A separate process
would be responsible for pulling operations off the queue and using
HBase client operations to do the insert into a secondary index table
appropriate for that operation.
     Perhaps we could use some of the work that the Lily people have
done with HBase indexing (see
http://www.lilyproject.org/lily/about/playground/hbaseindexes.html) in
order to do the edit->hbase operation transformations and the secondary
index table creation.
     -Eugene
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB