Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - key design


Copy link to this message
-
Re: key design
Otis Gospodnetic 2012-05-22, 21:14
Mete,

Have you tried https://github.com/sematext/HBaseWD ?  WD stands for Writes Distributor.

For a clear introductory post with code samples read: http://blog.sematext.com/2012/04/09/hbasewd-avoid-regionserver-hotspotting-despite-writing-records-with-sequential-keys/

Otis
----
Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm

>________________________________
> From: mete <[EMAIL PROTECTED]>
>To: [EMAIL PROTECTED]
>Sent: Monday, May 21, 2012 10:56 AM
>Subject: key design
>
>Hello folks,
>
>i am trying to come up with a nice key design for storing logs in the
>company. I am planning to index them  and store row key in the index for
>random reads.
>
>I need to balance the writes equally between the R.S. and i could not
>understand how opentsdb does that with prefixing the metric id. (i related
>metric id with the log type) In my log storage case a log line just has a
>type and a date and the rest of it is not really very useful information.
>
>So i think that i can create a table for every distinct log type and i need
>a random salt to route to a different R.S. similar to this:
><salt>-<date>
>
>But with this approach i believe i will lose the ability to do effective
>partial scans to a specific date. (if for some reason i need that) What do
>you think? And for the salt approach do you use randomly generated salts or
>hashes that actually mean something? (like the hash of the date)
>
>I am using random uuids at the moment but i am trying to find a better
>approach, any feedback is welcome
>
>cheers
>Mete
>
>
>