Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Suspected memory leak


Copy link to this message
-
Re: Suspected memory leak
Jieshan,
We backported https://issues.apache.org/jira/browse/HBASE-2937 to 0.90.3

-Shrijeet
2011/11/30 bijieshan <[EMAIL PROTECTED]>

> Hi Shrijeet,
>
> I think that's jira relevant to trunk, but not for 90.X. For there's no
> timeout mechanism in 90.X. Right?
> We found this problem in 90.x.
>
> Thanks,
>
> Jieshan.
>
> -----邮件原件-----
> 发件人: Shrijeet Paliwal [mailto:[EMAIL PROTECTED]]
> 发送时间: 2011年12月1日 10:26
> 收件人: [EMAIL PROTECTED]
> 抄送: Gaojinchao; Chenjian
> 主题: Re: Suspected memory leak
>
> Gaojinchao,
>
> I had filed this some time ago,
> https://issues.apache.org/jira/browse/HBASE-4633
> But after some recent insights on our application code, I am inclined to
> think leak (or memory 'hold') is in our application. But it will be good to
> check out either way.
> I need to update the jira with my saga. See if the description of issue I
> posted there, matches yours. If not, may be you can update with your story
> in detail.
>
> -Shrijeet
>
> 2011/11/30 Gaojinchao <[EMAIL PROTECTED]>
>
> > I have noticed some memory leak problems in my HBase client.
> > RES has increased to 27g
> > PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> > 12676 root      20   0 30.8g  27g 5092 S    2 57.5 587:57.76
> > /opt/java/jre/bin/java -Djava.library.path=lib/.
> >
> > But I am not sure the leak comes from HBase Client jar itself or just our
> > client code.
> >
> > This is some parameters of jvm.
> > :-Xms15g -Xmn12g -Xmx15g -XX:PermSize=64m -XX:+UseParNewGC
> > -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=65
> > -XX:+UseCMSCompactAtFullCollection -XX:CMSFullGCsBeforeCompaction=1
> > -XX:+CMSParallelRemarkEnabled
> >
> > Who has experience in this case? , I need continue to dig :)
> >
> >
> >
> > 发件人: Gaojinchao
> > 发送时间: 2011年11月30日 11:02
> > 收件人: [EMAIL PROTECTED]
> > 主题: Suspected memory leak
> >
> > In HBaseClient proceess, I found heap has been increased.
> > I used command ’cat smaps’ to get the heap size.
> > It seems in case when the threads pool in HTable has released the no
> using
> > thread, if you use putlist api to put data again, the memory is
> increased.
> >
> > Who has experience in this case?
> >
> > Below is the heap of Hbase client:
> > C3S31:/proc/18769 # cat smaps
> > 4010a000-4709d000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             114252 kB
> > Rss:              114044 kB
> > Pss:              114044 kB
> >
> > 4010a000-4709d000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             114252 kB
> > Rss:              114044 kB
> > Pss:              114044 kB
> >
> > 4010a000-48374000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             133544 kB
> > Rss:              133336 kB
> > Pss:              133336 kB
> >
> > 4010a000-49f20000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             161880 kB
> > Rss:              161672 kB
> > Pss:              161672 kB
> >
> > 4010a000-4c5de000 rwxp 00000000 00:00 0
> >  [heap]
> > Size:             201552 kB
> > Rss:              201344 kB
> > Pss:              201344 kB
> >
>