New zookeeper server fails to join quorum with msg "Have smaller server identifie"
I replacing one of the zookeeper server from 3 node quorum.
Initially all zookeeper serves were running 22.214.171.1245976 version.
I successfully replaced Node3 with newer version 126.96.36.1991730.
When I am trying to replace Node2 with the same zookeeper version.
I couldn't start zookeeper server on Node2 as it is continuously stuck in
leader election loop printing following messages:
2014-02-26 02:45:23,709 [myid:3] - INFO
Notification time out: 60000
2014-02-26 02:45:23,710 [myid:3] - INFO
[WorkerSender[myid=3]:QuorumCnxManager@195] - Have smaller server
identifier, so dropping the connection: (5, 3)
2014-02-26 02:45:23,712 [myid:3] - INFO
[WorkerReceiver[myid=3]:FastLeaderElection@605] - Notification: 3
(n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0
(n.peerEPoch), LOOKING (my state)1 (n.config version)
Network connections and configuration of the node being upgraded are fine.
The other 2 nodes in the quorum are fine and serving the request.
Any idea what might be causing this?
Thanks & Regards,