Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Controlling Region Server Heap usage


Copy link to this message
-
Controlling Region Server Heap usage
Hi all,

I'm working on a production cluster where each Region Server has 10GB
allocated memory, and I'm trying to control (limit) the memory usage during
massive writes.

HBase configurations are:
hbase.regionserver.handler.count 18
hbase.regionserver.global.memstore.upperLimit 0.5
hbase.regionserver.global.memstore.lowerLimit 0.45
hbase.server.thread.wakefrequency 500
hbase.hregion.memstore.flush.size 268435456 (256MB)
hbase.hregion.memstore.block.multiplier 5
hbase.hstore.blockingStoreFiles 12

As I understand things, when the heap usage of a Region Server reaches
the hbase.regionserver.global.memstore.upperLimit (5GB in this case), all
updates are blocked and all MemStores are flushed (until lowerLimit is
reached - 4.5GB in this case).

During massive writes to HBase I see some of the Region Servers constantly
(not just spikes) over 7GB and spiking to 9 from time to time.

Why is That ? is there anything wrong with the configurations I used ?
Is there a better way to control the Region Server memory usage ?

Thanks,

Amit.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB