Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Re: Suspected memory leak


+
Gaojinchao 2011-12-01, 02:13
Copy link to this message
-
Re: Suspected memory leak
Shrijeet Paliwal 2011-12-01, 02:25
Gaojinchao,

I had filed this some time ago,
https://issues.apache.org/jira/browse/HBASE-4633
But after some recent insights on our application code, I am inclined to
think leak (or memory 'hold') is in our application. But it will be good to
check out either way.
I need to update the jira with my saga. See if the description of issue I
posted there, matches yours. If not, may be you can update with your story
in detail.

-Shrijeet

2011/11/30 Gaojinchao <[EMAIL PROTECTED]>

> I have noticed some memory leak problems in my HBase client.
> RES has increased to 27g
> PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 12676 root      20   0 30.8g  27g 5092 S    2 57.5 587:57.76
> /opt/java/jre/bin/java -Djava.library.path=lib/.
>
> But I am not sure the leak comes from HBase Client jar itself or just our
> client code.
>
> This is some parameters of jvm.
> :-Xms15g -Xmn12g -Xmx15g -XX:PermSize=64m -XX:+UseParNewGC
> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=65
> -XX:+UseCMSCompactAtFullCollection -XX:CMSFullGCsBeforeCompaction=1
> -XX:+CMSParallelRemarkEnabled
>
> Who has experience in this case? , I need continue to dig :)
>
>
>
> 发件人: Gaojinchao
> 发送时间: 2011年11月30日 11:02
> 收件人: [EMAIL PROTECTED]
> 主题: Suspected memory leak
>
> In HBaseClient proceess, I found heap has been increased.
> I used command ’cat smaps’ to get the heap size.
> It seems in case when the threads pool in HTable has released the no using
> thread, if you use putlist api to put data again, the memory is increased.
>
> Who has experience in this case?
>
> Below is the heap of Hbase client:
> C3S31:/proc/18769 # cat smaps
> 4010a000-4709d000 rwxp 00000000 00:00 0
>  [heap]
> Size:             114252 kB
> Rss:              114044 kB
> Pss:              114044 kB
>
> 4010a000-4709d000 rwxp 00000000 00:00 0
>  [heap]
> Size:             114252 kB
> Rss:              114044 kB
> Pss:              114044 kB
>
> 4010a000-48374000 rwxp 00000000 00:00 0
>  [heap]
> Size:             133544 kB
> Rss:              133336 kB
> Pss:              133336 kB
>
> 4010a000-49f20000 rwxp 00000000 00:00 0
>  [heap]
> Size:             161880 kB
> Rss:              161672 kB
> Pss:              161672 kB
>
> 4010a000-4c5de000 rwxp 00000000 00:00 0
>  [heap]
> Size:             201552 kB
> Rss:              201344 kB
> Pss:              201344 kB
>
+
bijieshan 2011-12-01, 02:29
+
Shrijeet Paliwal 2011-12-01, 02:38
+
bijieshan 2011-12-01, 02:44
+
Gaojinchao 2011-11-30, 03:01