Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> Connection broken for id 2, error = EOFException


Copy link to this message
-
Connection broken for id 2, error = EOFException
*Hi List,*
*
*
*I'm using ZooKeeper 3.4.3 with jdk 1.6 on a RHEL 5.5 box. On one of the
nodes I constantly get the following warning logs:*

2013-01-04 16:42:15,458 [myid:1] - WARN
 [RecvWorker:2:QuorumCnxManager$RecvWorker@762] - Connection broken for id
2, my id = 1, error java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at
org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:747)
2013-01-04 16:42:15,458 [myid:1] - WARN
 [RecvWorker:2:QuorumCnxManager$RecvWorker@765] - Interrupting SendWorker
2013-01-04 16:42:15,458 [myid:1] - WARN
 [SendWorker:2:QuorumCnxManager$SendWorker@679] - Interrupted while waiting
for message on queue
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2038)
at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:340)
at
org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:831)
at
org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:62)
at
org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:667)
2013-01-04 16:42:15,459 [myid:1] - WARN
 [SendWorker:2:QuorumCnxManager$SendWorker@688] - Send worker leaving thread

*And the corresponding log on node id 2 is:*

2013-01-04 16:42:15,063 [myid:2] - INFO
 [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 2
(n.leader), 0x500000004 (n.zxid), 0x6 (n.round), LOOKING (n.state), 2
(n.sid), 0x5 (n.peerEPoch), FOLLOWING (my state)
2013-01-04 16:42:15,064 [myid:2] - INFO
 [WorkerReceiver[myid=2]:FastLeaderElection@542] - Notification: 3
(n.leader), 0x700036b0d (n.zxid), 0xb (n.round), FOLLOWING (n.state), 2
(n.sid), 0x7 (n.peerEPoch), FOLLOWING (my state)

*These two nodes are both in follower mode.*
*
*
*Besides I also constantly get the following exception:*

2013-01-04 16:46:34,439 [myid:1] - WARN  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@349] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid
0x13c04b044050014, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:224)
at java.lang.Thread.run(Thread.java:662)

*I try to trace the sessionid, it seems that all the sessions are from the
applications running on id 1 box (We do not use dedicated server for
zookeeper clusters.)*
*
*
*Any ideas?*
*
*
*Thanks.*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB