Hi all, i read in documentation that max region count per region server should be around 200. I unfortunately have currently 800 regions for RS. Those tables are not presplitted (all handled by hbase), does manual spliting helps in this case? hdfs fodler /hbase/data currently hold around 2.2TB of data and i have 4 region servers, is it enough or i need more RS?
Also will Hbase compact these tables automatically to have less regions?
Hbase version 0.96.1.1-cdh5.0.1 Thanks for any guidance.
So you have 3200 regions in total, and that means each region is about 650mb in size only (if you have 2.2 Tb in total), unless I'm missing something? If that's the case your regions are probably way too small.
-Mikhail 2014-08-15 0:44 GMT-07:00 Juraj jiv <[EMAIL PROTECTED]>: Thanks, Michael Antonov
HBase initially tries to spread the load out to more region servers by splitting regions early when there aren't many (per table). As more regions are seen split size approaches the configured max file size. This early splitting turned out to be a bit too aggressive (see HBASE-10501, which modifies it a bit)... Even then 800 regions is a lot.
As others have asked, what's
hbase.hregion.max.filesize set to? Make sure it's at least 10gb.
________________________________ From: Juraj jiv <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Sent: Friday, August 15, 2014 2:08 AM Subject: Re: Hbase region count and RS count for 2TB+
Hi, size 2.2TB i got from cmd "hdfs dfs -du -h /hbase/data"
So i think yes, 650MB per RS. If its small then why hbase created so many regions?
On Fri, Aug 15, 2014 at 9:51 AM, Mikhail Antonov <[EMAIL PROTECTED]> wrote:
Hello all, thanks for replies. hbase.hregion.max.filesize is set to default size 1GB. I set it to 10GB as recommended.
JV On Fri, Aug 15, 2014 at 9:15 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext