Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Java Commited Virtual Memory significally larged then Heap Memory


Copy link to this message
-
Re: Java Commited Virtual Memory significally larged then Heap Memory
with MALLOC_ARENA_MAX=2

I check -XX:MaxDirectMemorySize=256m, before, but it doesn't affect anything
(even no OOM
exceptions or so on).

But it looks like i have exactly the same issue (it looks like). I have many
64Mb anon memory blocks.
(sometimes they 132MB). And on heavy load i have rapidly growing rss size of
jvm process.

2011/1/12 Friso van Vollenhoven <[EMAIL PROTECTED]>

> Just to clarify: you fixed it by setting the MALLOC_MAX_ARENA=? in
> hbase-env.sh?
>
> Did you also use the -XX:MaxDirectMemorySize=256m ?
>
> It would be nice to check that this is a different than the leakage with
> LZO...
>
>
> Thanks,
> Friso
>
>
> On 12 jan 2011, at 07:46, Andrey Stepachev wrote:
>
> > My bad. All things work. Thanks for  Todd Lipcon :)
> >
> > 2011/1/11 Andrey Stepachev <[EMAIL PROTECTED]>
> >
> >> I tried to set MALLOC_ARENA_MAX=2. But still the same issue like in LZO
> >> problem thread. All those 65M blocks here. And JVM continues to eat
> memory
> >> on heavy write load. And yes, I use "improved" kernel
> >> Linux 2.6.34.7-0.5.
> >>
> >> 2011/1/11 Xavier Stevens <[EMAIL PROTECTED]>
> >>
> >> Are you using a newer linux kernel with the new and "improved" memory
> >>> allocator?
> >>>
> >>> If so try setting this in hadoop-env.sh:
> >>>
> >>> export MALLOC_ARENA_MAX=<number of cores you want to use>
> >>>
> >>> Maybe start by setting it to 4.  You can thank Todd Lipcon if this
> works
> >>> for you.
> >>>
> >>> Cheers,
> >>>
> >>>
> >>> -Xavier
> >>>
> >>> On 1/11/11 7:24 AM, Andrey Stepachev wrote:
> >>>> No. I don't use LZO. I tried even remove any native support (i.e. all
> >>> .so
> >>>> from class path)
> >>>> and use java gzip. But nothing.
> >>>>
> >>>>
> >>>> 2011/1/11 Friso van Vollenhoven <[EMAIL PROTECTED]>
> >>>>
> >>>>> Are you using LZO by any chance? If so, which version?
> >>>>>
> >>>>> Friso
> >>>>>
> >>>>>
> >>>>> On 11 jan 2011, at 15:57, Andrey Stepachev wrote:
> >>>>>
> >>>>>> After starting the hbase in jroсkit found the same memory leakage.
> >>>>>>
> >>>>>> After the launch
> >>>>>>
> >>>>>> Every 2,0 s: date & & ps - sort =- rss-eopid, rss, vsz, pcpu | head
> >>>>>> Tue Jan 11 16:49:31 2011
> >>>>>>
> >>>>>>  11 16:49:31 MSK 2011
> >>>>>>  PID RSS VSZ% CPU
> >>>>>> 7863 2547760 5576744 78.7
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> JR dumps:
> >>>>>>
> >>>>>> Total mapped 5576740KB (reserved = 2676404KB) - Java heap 2048000KB
> >>>>>> (reserved = 1472176KB) - GC tables 68512KB - Thread stacks 37236KB
> (#
> >>>>>> threads = 111) - Compiled code 1048576KB (used = 2599KB) - Internal
> >>>>>> 1224KB - OS 549688KB - Other 1800976KB - Classblocks 1280KB
> (malloced
> >>>>>> = 1110KB # 3285) - Java class data 20224KB (malloced = 20002KB #
> 15134
> >>>>>> in 3285 classes) - Native memory tracking 1024KB (malloced = 325KB
> +10
> >>>>>> KB # 20)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> After running the mr which make high write load (~1hour)
> >>>>>>
> >>>>>> Every 2,0 s: date & & ps - sort =- rss-eopid, rss, vsz, pcpu | head
> >>>>>> Tue Jan 11 17:08:56 2011
> >>>>>>
> >>>>>>  11 17:08:56 MSK 2011
> >>>>>>  PID RSS VSZ% CPU
> >>>>>> 7863 4072396 5459572 100
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> JR said not important below specify why)
> >>>>>>
> >>>>>> http://paste.ubuntu.com/552820/
> >>>>>> <http://paste.ubuntu.com/552820/>
> >>>>>>
> >>>>>>
> >>>>>> 7863:
> >>>>>> Total mapped                  5742628KB +165888KB
> (reserved=1144000KB
> >>>>>> -1532404KB)
> >>>>>> -              Java heap      2048000KB           (reserved=0KB
> >>>>> -1472176KB)
> >>>>>> -              GC tables        68512KB
> >>>>>> -          Thread stacks        38028KB    +792KB (#threads=114 +3)
> >>>>>> -          Compiled code      1048576KB           (used=3376KB
> +776KB)
> >>>>>> -               Internal         1480KB    +256KB
> >>>>>> -                     OS       517944KB  -31744KB
> >>>>>> -                  Other      1996792KB +195816KB
> >>>>>> -            Classblocks         1280KB           (malloced=1156KB
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB