Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper, mail # user - Memory leak caused by a watch that will never be triggered

David Nickerson 2012-05-30, 18:35
Copy link to this message
Re: Memory leak caused by a watch that will never be triggered
Mark Gius 2012-05-30, 18:56
What language are you using?  Does it have some equivalent of Python's
http://docs.python.org/library/weakref.html?  If so you can pass this weak
reference to the watcher (and have some basic code failing gracefully if
the referred object has been purged already).


On Wed, May 30, 2012 at 11:35 AM, David Nickerson <

> I writing a lock implementation that agrees with the lock recipe in the
> documentation<
> http://zookeeper.apache.org/doc/current/recipes.html#sc_recipes_Locks>.
> In step 5, the thread needs to wait for a notification that it may have the
> lock. Since I'm working with many threads, each thread needs to wait on its
> own object. I decided that it would be easiest if the thread waits on the
> watcher that is passed as a parameter to zk.exists in step 4. This way,
> when a thread may have the lock, its watcher gets called directly.
> (The other option would be to have only one watcher. Then, when
> watcher.process() gets called, the watcher would need to figure out which
> thread it should wake up.)
> The only problem is that I might have a memory leak. In step 5, if exists()
> returns false, then the node does not exist and it will never exist because
> it is sequential. This means that the client will forever hold a watch for
> a node that will never be created, and that watch will forever reference my
> watcher object. Therefore, even if I try to dump that watcher object (since
> I don't need it anymore), the client will still hold a reference to it and
> it won't be garbage collected.
> The only way that I've come up with to plug the memory leak is to recreate
> the znode as non-sequential, thereby triggering the watch I set, and then
> delete the znode to prevent a deadlock.
> Can anyone confirm that this is indeed a memory leak, and does anyone know
> a better way to prevent it?
David Nickerson 2012-05-30, 19:11
David Nickerson 2012-06-01, 15:36
David Nickerson 2012-06-15, 14:05