-Re: Two Leaders?!
Marshall McMullen 2011-12-20, 19:21
What specific log files should I look for?
I inspected the config files for all 3 nodes and they *are different.
the servers specified are not consistent:
$ cat /data/zookeeper/10.10.5.56/10.10.5.56_2181.cfg
$ cat /data/zookeeper/10.10.5.58/10.10.5.58_2181.cfg
$ cat /data/zookeeper/10.10.5.46/10.10.5.46_2181.cfg
So this looks like a configuration problem not a zookeeper bug correct?
On Tue, Dec 20, 2011 at 11:17 AM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
> Really the logs are critical here. If you can provide them it would shed
> On Tue, Dec 20, 2011 at 10:13 AM, Benjamin Reed <[EMAIL PROTECTED]> wrote:
> > i've seen it before when the configuration files haven't been setup
> > properly. i would check the configuration. if the leader is still the
> > leader, it must have active followers connected to it, otherwise it
> > would give up leadership. i would use netstat to find out who they
> > are.
> > ben
> > On Tue, Dec 20, 2011 at 9:00 AM, Marshall McMullen
> > <[EMAIL PROTECTED]> wrote:
> >> Zookeeper devs,
> >> I've got a cluster with 3 servers in the ensemble all running 3.4.0.
> >> a few days of successful operation, we observed all zookeeper reads and
> >> writes began failing every time. In our log files, the error being
> >> is INVALID_STATE. I then telnetted to port 2181 on all three servers and
> >> was surprised to see that *two* of these servers both report they are
> >> leader! Two of the nodes are in agreement on the Zxid, and one of the
> >> is way out of whack with a much much larger Zxid. The node that all
> >> are flowing through is the one with the much higher Zxid.
> >> Has anyone ever seen this before? What can I do to diagnose this problem
> >> and resolve it? I was considering killing zookeeper on the node that
> >> not be the leader (the one with the wrong Zxid) and removing the
> >> data directory, then restarting zookeeper on that node. Any other ideas?
> >> I appreciate any help.