Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Region server Memory Use is double the -Xmx setting


Copy link to this message
-
RE: Region server Memory Use is double the -Xmx setting
Are  you using compression for HFiles?

Yes we are using  "MaxDirectMemorySize" and we dont use off-heap cache.

-Anoop-
________________________________________
From: Buckley,Ron [[EMAIL PROTECTED]]
Sent: Wednesday, January 23, 2013 8:49 PM
To: [EMAIL PROTECTED]
Subject: RE: Region server Memory Use is double the -Xmx setting

Liang,

Thanks.  I wasn’t really aware that the direct memory could get that large. (Full disclosure, we did switch from jdk1.6.0_25 to jdk1.6.0_31 the last time we restarted HBase.)

I've only seen explicit setting of -XX:MaxDirectMemorySize for regionservers associated with the experimental off-heap cache.

Is anyone else running their region servers with -XX:MaxDirectMemorySize (not using the off-heap cache)?

Ron

-----Original Message-----
From: 谢良 [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, January 22, 2013 9:20 PM
To: [EMAIL PROTECTED]
Subject: 答复: Region server Memory Use is double the -Xmx setting

Please set "-XX:MaxDirectMemorySize" explicitly,  else the default is taking the value like -Xmx in currenty JDK6, at least for jdk1.6.30+

Best Regards,
Liang
________________________________________
发件人: Buckley,Ron [[EMAIL PROTECTED]]
发送时间: 2013年1月23日 5:17
收件人: [EMAIL PROTECTED]
主题: Region server Memory Use is double the -Xmx setting

We have a 50 node cluster replicating to a 6 node cluster. Both clusters
are running CDH4.1.2 and HBase 0.94.2.

Today we noticed that the region servers at our replica site are using
10GB more memory than the '-Xmx12288m' we have defined in hbase-env.sh

These region servers have been up since January 9, 2013.

Does anyone have suggestions about tracking down this additional memory
use?

I'm not necessarily expecting the Region Server to stay right at the
12GB that we allocated, but having it running at 24GB is starting to
cause the servers to swap.
PID    USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
28544  prodcon   18   0 24.1g  23g  18m S 20.0 74.0   9071:34 java
28544:   /usr/java/jdk1.6.0_31/bin/java -XX:OnOutOfMemoryError=kill -9
%p -Xmx12288m -Dcom.sun.management.jmxr

emote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.port=9021 -ea

-server -XX:+HeapDumpOnOutOfMemoryError -Xmn256m -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOc

cupancyFraction=70 -verbose:gc -XX:+PrintGCDetails
-XX:+PrintGCDateStamps -Xloggc:/drive1/hadoop/2.0/isoft/../

hbase/logs/gc-hbase.log -ea -server -XX:+HeapDumpOnOutOfMemoryError
-Xmn256m -XX

----------------------------------------------------------------------

Ron Buckley

x6365

http://intranet-wiki.oclc.org/wiki/XWC/XServe/SRU

http://intranet-wiki.oclc.org/wiki/FIND

http://intranet-wiki.oclc.org/wiki/Higgins
http://intranet-wiki.oclc.org/wiki/Firefly