Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> key design


Mete,

Have you tried https://github.com/sematext/HBaseWD ?  WD stands for Writes Distributor.

For a clear introductory post with code samples read: http://blog.sematext.com/2012/04/09/hbasewd-avoid-regionserver-hotspotting-despite-writing-records-with-sequential-keys/

Otis
----
Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm

>________________________________
> From: mete <[EMAIL PROTECTED]>
>To: [EMAIL PROTECTED]
>Sent: Monday, May 21, 2012 10:56 AM
>Subject: key design
>
>Hello folks,
>
>i am trying to come up with a nice key design for storing logs in the
>company. I am planning to index them  and store row key in the index for
>random reads.
>
>I need to balance the writes equally between the R.S. and i could not
>understand how opentsdb does that with prefixing the metric id. (i related
>metric id with the log type) In my log storage case a log line just has a
>type and a date and the rest of it is not really very useful information.
>
>So i think that i can create a table for every distinct log type and i need
>a random salt to route to a different R.S. similar to this:
><salt>-<date>
>
>But with this approach i believe i will lose the ability to do effective
>partial scans to a specific date. (if for some reason i need that) What do
>you think? And for the salt approach do you use randomly generated salts or
>hashes that actually mean something? (like the hash of the date)
>
>I am using random uuids at the moment but i am trying to find a better
>approach, any feedback is welcome
>
>cheers
>Mete
>
>
>  
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB