Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - one table flushes at much smaller sizes than other?

Norbert Burger 2013-12-26, 21:52
Copy link to this message
答复: one table flushes at much smaller sizes than other?
谢良 2013-12-27, 03:51
Do you observe some log like "Too many hlogs" before flushing a few
dozen KB log ?  Just want to double-check it's weather a forcing flush issue
or not:)

发件人: Norbert Burger [[EMAIL PROTECTED]]
发送时间: 2013年12月27日 5:52
主题: one table flushes at much smaller sizes than other?

Hi folks, we're currently running CDH4.1.2 (HBase 0.92 + patches).

Looking through the logs, I'm seeing that some tables are flushing at much
lower size than others (a few dozen KB as opposed to the expected 128 MB).

Looking through some of great case studies like [1], [2], and [3], it's not
clear to me why this is happening.  We're using the defaults for upper,
lower limits (0.35 and 0.40, respectively), and the RS status pages show
that we're only using 25% of our memstore allocation.

Our regions/RS ratio is currently at 60-70.  We do have some other tables
online where column families are skewed heavily sizewise -- could this be
causing other tables in our cluster to flush more frequently than others?


[1] http://blog.sematext.com/2012/07/16/hbase-memstore-what-you-should-know/
[2] http://www.ngdata.com/visualizing-hbase-flushes-and-compactions/
[3] http://gbif.blogspot.com/2012/07/optimizing-writes-in-hbase.html
Ted Yu 2013-12-27, 04:03
Norbert Burger 2013-12-27, 04:35
Ted Yu 2013-12-27, 05:03
Norbert Burger 2013-12-27, 17:44
Ted Yu 2013-12-27, 21:08