-Re: zkclient timeout issue
Patrick Hunt 2013-10-14, 20:29
Are you sure that the JVM is not swapping? Perhaps the "disk intensive
process" is also eating up alot of memory and that's pushing ZK server
out to swap?
Turn swappyness off and retry.
On Thu, Oct 10, 2013 at 10:31 AM, Srikanth R <[EMAIL PROTECTED]> wrote:
> Hi Guys,
> Need some expert advice :)
> I have a 3 Server Zookeeper ensemble. Only one client (hadoop-zkfc with
> 5second timeout) is connected to zookeeper. There is no other activity,
> one txn is recorded every 10 min in zookeeper txn log. Zookeeper is as
> close to being idle, except for the session traffic from the hadoop-zkfc
> Even with no writes happening to the zookeeper data dir, if I start some
> disk intensive process on the same partition that has datadir (like
> raid-check or cat'ing huge files), I am observing zkclient session
> timeouts. (Error: Client has not heard back from Server in 3334 ms, so
> disconnecting and reconnecting)
> strace on zookeeper process shows that it received the 12byte heartbeat
> from the client, but has not responded. Also strace does not show any disk
> activity from zoookeper.
> So the question is does zookeeper do anything on the disk even when its
> idle (not writing any txns to disk). why does the datadir disk utilization
> affect zookeeper even without any traffic ?
> Any help is appreciated in this regard.