Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Controlling Region Server Heap usage


Copy link to this message
-
Controlling Region Server Heap usage
Amit Sela 2012-06-13, 15:40
Hi all,

I'm working on a production cluster where each Region Server has 10GB
allocated memory, and I'm trying to control (limit) the memory usage during
massive writes.

HBase configurations are:
hbase.regionserver.handler.count 18
hbase.regionserver.global.memstore.upperLimit 0.5
hbase.regionserver.global.memstore.lowerLimit 0.45
hbase.server.thread.wakefrequency 500
hbase.hregion.memstore.flush.size 268435456 (256MB)
hbase.hregion.memstore.block.multiplier 5
hbase.hstore.blockingStoreFiles 12

As I understand things, when the heap usage of a Region Server reaches
the hbase.regionserver.global.memstore.upperLimit (5GB in this case), all
updates are blocked and all MemStores are flushed (until lowerLimit is
reached - 4.5GB in this case).

During massive writes to HBase I see some of the Region Servers constantly
(not just spikes) over 7GB and spiking to 9 from time to time.

Why is That ? is there anything wrong with the configurations I used ?
Is there a better way to control the Region Server memory usage ?

Thanks,

Amit.