Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Re: Suspected memory leak


+
Gaojinchao 2011-12-01, 02:13
+
Shrijeet Paliwal 2011-12-01, 02:25
+
bijieshan 2011-12-01, 02:29
Copy link to this message
-
Re: Suspected memory leak
Jieshan,
We backported https://issues.apache.org/jira/browse/HBASE-2937 to 0.90.3

-Shrijeet
2011/11/30 bijieshan <[EMAIL PROTECTED]>

> Hi Shrijeet,
>
> I think that's jira relevant to trunk, but not for 90.X. For there's no
> timeout mechanism in 90.X. Right?
> We found this problem in 90.x.
>
> Thanks,
>
> Jieshan.
>
> -----邮件原件-----
> 发件人: Shrijeet Paliwal [mailto:[EMAIL PROTECTED]]
> 发送时间: 2011年12月1日 10:26
> 收件人: [EMAIL PROTECTED]
> 抄送: Gaojinchao; Chenjian
> 主题: Re: Suspected memory leak
>
> Gaojinchao,
>
> I had filed this some time ago,
> https://issues.apache.org/jira/browse/HBASE-4633
> But after some recent insights on our application code, I am inclined to
> think leak (or memory 'hold') is in our application. But it will be good to
> check out either way.
> I need to update the jira with my saga. See if the description of issue I
> posted there, matches yours. If not, may be you can update with your story
> in detail.
>
> -Shrijeet
>
> 2011/11/30 Gaojinchao <[EMAIL PROTECTED]>
>
> > I have noticed some memory leak problems in my HBase client.
> > RES has increased to 27g
> > PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> > 12676 root      20   0 30.8g  27g 5092 S    2 57.5 587:57.76
> > /opt/java/jre/bin/java -Djava.library.path=lib/.
> >
> > But I am not sure the leak comes from HBase Client jar itself or just our
> > client code.
> >
> > This is some parameters of jvm.
> > :-Xms15g -Xmn12g -Xmx15g -XX:PermSize=64m -XX:+UseParNewGC
> > -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=65
> > -XX:+UseCMSCompactAtFullCollection -XX:CMSFullGCsBeforeCompaction=1
> > -XX:+CMSParallelRemarkEnabled
> >
> > Who has experience in this case? , I need continue to dig :)
> >
> >
> >
> > 发件人: Gaojinchao
> > 发送时间: 2011年11月30日 11:02
> > 收件人: [EMAIL PROTECTED]
> > 主题: Suspected memory leak
> >
> > In HBaseClient proceess, I found heap has been increased.
> > I used command ’cat smaps’ to get the heap size.
> > It seems in case when the threads pool in HTable has released the no
> using
> > thread, if you use putlist api to put data again, the memory is
> increased.
> >
> > Who has experience in this case?
> >
> > Below is the heap of Hbase client:
> > C3S31:/proc/18769 # cat smaps
> > 4010a000-4709d000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             114252 kB
> > Rss:              114044 kB
> > Pss:              114044 kB
> >
> > 4010a000-4709d000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             114252 kB
> > Rss:              114044 kB
> > Pss:              114044 kB
> >
> > 4010a000-48374000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             133544 kB
> > Rss:              133336 kB
> > Pss:              133336 kB
> >
> > 4010a000-49f20000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             161880 kB
> > Rss:              161672 kB
> > Pss:              161672 kB
> >
> > 4010a000-4c5de000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             201552 kB
> > Rss:              201344 kB
> > Pss:              201344 kB
> >
>
+
bijieshan 2011-12-01, 02:44
+
Gaojinchao 2011-11-30, 03:01
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB