Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Suspected memory leak


Copy link to this message
-
Re: Suspected memory leak
Gaojinchao,

I had filed this some time ago,
https://issues.apache.org/jira/browse/HBASE-4633
But after some recent insights on our application code, I am inclined to
think leak (or memory 'hold') is in our application. But it will be good to
check out either way.
I need to update the jira with my saga. See if the description of issue I
posted there, matches yours. If not, may be you can update with your story
in detail.

-Shrijeet

2011/11/30 Gaojinchao <[EMAIL PROTECTED]>

> I have noticed some memory leak problems in my HBase client.
> RES has increased to 27g
> PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 12676 root      20   0 30.8g  27g 5092 S    2 57.5 587:57.76
> /opt/java/jre/bin/java -Djava.library.path=lib/.
>
> But I am not sure the leak comes from HBase Client jar itself or just our
> client code.
>
> This is some parameters of jvm.
> :-Xms15g -Xmn12g -Xmx15g -XX:PermSize=64m -XX:+UseParNewGC
> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=65
> -XX:+UseCMSCompactAtFullCollection -XX:CMSFullGCsBeforeCompaction=1
> -XX:+CMSParallelRemarkEnabled
>
> Who has experience in this case? , I need continue to dig :)
>
>
>
> 发件人: Gaojinchao
> 发送时间: 2011年11月30日 11:02
> 收件人: [EMAIL PROTECTED]
> 主题: Suspected memory leak
>
> In HBaseClient proceess, I found heap has been increased.
> I used command ’cat smaps’ to get the heap size.
> It seems in case when the threads pool in HTable has released the no using
> thread, if you use putlist api to put data again, the memory is increased.
>
> Who has experience in this case?
>
> Below is the heap of Hbase client:
> C3S31:/proc/18769 # cat smaps
> 4010a000-4709d000 rwxp 00000000 00:00 0
>  [heap]
> Size:             114252 kB
> Rss:              114044 kB
> Pss:              114044 kB
>
> 4010a000-4709d000 rwxp 00000000 00:00 0
>  [heap]
> Size:             114252 kB
> Rss:              114044 kB
> Pss:              114044 kB
>
> 4010a000-48374000 rwxp 00000000 00:00 0
>  [heap]
> Size:             133544 kB
> Rss:              133336 kB
> Pss:              133336 kB
>
> 4010a000-49f20000 rwxp 00000000 00:00 0
>  [heap]
> Size:             161880 kB
> Rss:              161672 kB
> Pss:              161672 kB
>
> 4010a000-4c5de000 rwxp 00000000 00:00 0
>  [heap]
> Size:             201552 kB
> Rss:              201344 kB
> Pss:              201344 kB
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB