Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> ZooKeeper Memory Usage


Copy link to this message
-
Re: ZooKeeper Memory Usage
In my case, our stress test show up a linear increase of "tenured memory"
from 0 to > 3GiB with ZK 3.4.0 whereas the same stress-test with 3.3.3
keeps "tenured memory" stable and < 10MiB.

The stress test performs many zNodes creation and delete but the overall zk
usage at any moment in time was relative small.

BR,
/César.

On Thu, Feb 9, 2012 at 3:14 PM, Camille Fournier <[EMAIL PROTECTED]> wrote:

> This is really a question about how the jvm grows its heaps and resizes
> them. If the jvm cannot allocate enough memory for the process because you
> didn't set the max memory high enough, it will fall over. Zookeeper keeps
> its entire state in memory for performance reasons, if it were to swap that
> would be quite bad for performance.
>
> C
> On Feb 8, 2012 8:23 PM, "Mike Schilli" <[EMAIL PROTECTED]> wrote:
>
> > We've got a ZooKeeper instance that's using about 5 GB of resident
> > memory. Every time we restart it, it starts at 200MB, and then grows
> > slowly until it is back at 5 GB.
> >
> > The large footprint is related to how much data we've got in there.
> > What's interesting, though, is that the process size doesn't shrink if
> > we purge some of the data.
> >
> > Now, this isn't a big problem, I'm just curious if the process will fall
> > over at some point if it can't get more memory or if it'll just make due
> > by caching less data.
> >
> > Also, if I remember correctly, there's a configuration variable to set
> > the maximum size, what happens if ZK reaches that?
> >
> > -- -- Mike
> >
> > Mike Schilli
> > [EMAIL PROTECTED]
> >
>