Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Java Commited Virtual Memory significally larged then Heap Memory


Copy link to this message
-
Re: Java Commited Virtual Memory significally larged then Heap Memory
Andrey Stepachev 2011-01-11, 20:55
I tried to set MALLOC_ARENA_MAX=2. But still the same issue like in LZO
problem thread. All those 65M blocks here. And JVM continues to eat memory
on heavy write load. And yes, I use "improved" kernel
Linux 2.6.34.7-0.5.

2011/1/11 Xavier Stevens <[EMAIL PROTECTED]>

> Are you using a newer linux kernel with the new and "improved" memory
> allocator?
>
> If so try setting this in hadoop-env.sh:
>
> export MALLOC_ARENA_MAX=<number of cores you want to use>
>
> Maybe start by setting it to 4.  You can thank Todd Lipcon if this works
> for you.
>
> Cheers,
>
>
> -Xavier
>
> On 1/11/11 7:24 AM, Andrey Stepachev wrote:
> > No. I don't use LZO. I tried even remove any native support (i.e. all .so
> > from class path)
> > and use java gzip. But nothing.
> >
> >
> > 2011/1/11 Friso van Vollenhoven <[EMAIL PROTECTED]>
> >
> >> Are you using LZO by any chance? If so, which version?
> >>
> >> Friso
> >>
> >>
> >> On 11 jan 2011, at 15:57, Andrey Stepachev wrote:
> >>
> >>> After starting the hbase in jroсkit found the same memory leakage.
> >>>
> >>> After the launch
> >>>
> >>> Every 2,0 s: date & & ps - sort =- rss-eopid, rss, vsz, pcpu | head
> >>> Tue Jan 11 16:49:31 2011
> >>>
> >>>   11 16:49:31 MSK 2011
> >>>   PID RSS VSZ% CPU
> >>>  7863 2547760 5576744 78.7
> >>>
> >>>
> >>>
> >>> JR dumps:
> >>>
> >>> Total mapped 5576740KB (reserved = 2676404KB) - Java heap 2048000KB
> >>> (reserved = 1472176KB) - GC tables 68512KB - Thread stacks 37236KB (#
> >>> threads = 111) - Compiled code 1048576KB (used = 2599KB) - Internal
> >>> 1224KB - OS 549688KB - Other 1800976KB - Classblocks 1280KB (malloced
> >>> = 1110KB # 3285) - Java class data 20224KB (malloced = 20002KB # 15134
> >>> in 3285 classes) - Native memory tracking 1024KB (malloced = 325KB +10
> >>> KB # 20)
> >>>
> >>>
> >>>
> >>> After running the mr which make high write load (~1hour)
> >>>
> >>> Every 2,0 s: date & & ps - sort =- rss-eopid, rss, vsz, pcpu | head
> >>> Tue Jan 11 17:08:56 2011
> >>>
> >>>   11 17:08:56 MSK 2011
> >>>   PID RSS VSZ% CPU
> >>>  7863 4072396 5459572 100
> >>>
> >>>
> >>>
> >>> JR said not important below specify why)
> >>>
> >>> http://paste.ubuntu.com/552820/
> >>> <http://paste.ubuntu.com/552820/>
> >>>
> >>>
> >>> 7863:
> >>> Total mapped                  5742628KB +165888KB (reserved=1144000KB
> >>> -1532404KB)
> >>> -              Java heap      2048000KB           (reserved=0KB
> >> -1472176KB)
> >>> -              GC tables        68512KB
> >>> -          Thread stacks        38028KB    +792KB (#threads=114 +3)
> >>> -          Compiled code      1048576KB           (used=3376KB +776KB)
> >>> -               Internal         1480KB    +256KB
> >>> -                     OS       517944KB  -31744KB
> >>> -                  Other      1996792KB +195816KB
> >>> -            Classblocks         1280KB           (malloced=1156KB
> >>> +45KB #3421 +136)
> >>> -        Java class data        20992KB    +768KB (malloced=20843KB
> >>> +840KB #15774 +640 in 3421 classes)
> >>> - Native memory tracking         1024KB           (malloced=325KB +10KB
> >> #20)
> >>>
> >>>
> >>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>>    OS                          *java    r x 0x0000000000400000.(
> >> 76KB)
> >>>    OS                          *java    rw  0x0000000000612000 (
> >>  4KB)
> >>>    OS                        *[heap]    rw  0x0000000000613000.(
> >> 478712KB)
> >>>   INT                           Poll    r   0x000000007fffe000 (
> >>  4KB)
> >>>   INT                         Membar    rw  0x000000007ffff000.(
> >>  4KB)
> >>>   MSP              Classblocks (1/2)    rw  0x0000000082ec0000 (
> >>  768KB)
> >>>   MSP              Classblocks (2/2)    rw  0x0000000082f80000 (
> >>  512KB)
> >>>  HEAP                      Java heap    rw
> >>  0x0000000083000000.(2048000KB)
> >>>                                         rw  0x00007f2574000000 (
> >>  65500KB)
> >>>                                             0x00007f2577ff7000.(