|
charles charles
2012-09-24, 23:52
charles charles
2012-09-24, 23:59
ramachandra
2012-09-25, 10:48
charles charles
2012-10-04, 09:17
Patrick Hunt
2012-10-06, 14:41
charles charles
2012-10-09, 11:49
|
-
RE: About initLimit and syncLimitcharles charles 2012-09-24, 23:52
Thx, kathleen. This is my understanding. If I config initLimit=10 and syncLimit=3, so as follow: 1. Leader will wait for follow 10*tickTime to do such things: connect to leader, and sync the lasted data from leader. But, if the time follow load snap data from local disk is included in 10*tickTime?
Load snap data from local disk Conn to leader Sync lasted data from leader A: load snap data from local disk,B: conn to leaderC: sync lasted data from leader. Maybe, you can tell me, 10*tickTime includes whick part? A or B+C or A+B+C 2. While in runtime, if follow's state too far behind a leadr, even to 3*tickTime, the follow will be dropped. Others, please check my understanding, thx! Charles > Date: Mon, 24 Sep 2012 10:19:00 -0500 > Subject: Re: About initLimit and syncLimit > From: [EMAIL PROTECTED] > To: [EMAIL PROTECTED] > > Charles, > > initLimit measures how much time we anticipate the bootstrapping to > need whereas syncLimit controls how far behind the Followers can fall > behind the Leader. Sounds like you might be hitting ZOOKEEPER-1521. As > a result, can you try increasing both initLimit and syncLimit on all > ZK servers and then restarting? > > Regards, Kathleen > > On Mon, Sep 24, 2012 at 8:30 AM, charles charles > <[EMAIL PROTECTED]> wrote: > > initLimit > > Amount of time, in ticks (see tickTime), to allow followers to connect and sync to a leader. Increased this value as needed, if the amount of data managed by ZooKeeper is large. > > > > syncLimit > > Amount of time, in ticks (see tickTime), to allow followers to sync with ZooKeeper. If followers fall too far behind a leader, they will be dropped. In zookeeper config file zoo.cfg, we can config above two parameters, but just from above description, I can't know the difference between initLimit and syncLimit. Anyone can help me, tell me more about them, since my server can't come up, maybe, reson in in these two parameters. > > > > Thx! Charles > > > From: [EMAIL PROTECTED] > >> To: [EMAIL PROTECTED] > >> Subject: About initLimit and syncLimit > >> Date: Mon, 24 Sep 2012 06:25:53 -0700 > >> > >> initLimitAmount of time, in ticks (see tickTime), to allow followers to connect and sync to a leader. Increased this value as needed, if the amount of data managed by ZooKeeper is large.syncLimitAmount of time, in ticks (see tickTime), to allow followers to sync with ZooKeeper. If followers fall too far behind a leader, they will be dropped. In zookeeper config file zoo.cfg, we can config above two parameters, but just from above description, I can't know the difference between initLimit and syncLimit. Anyone can help me, tell me more about them, since my server can't come up, maybe, reson in in these two parameters. Thx! Charles > > +
charles charles 2012-09-24, 23:52
-
RE: About initLimit and syncLimitcharles charles 2012-09-24, 23:59
Oh,I'm sorry for my dirty format! Thx, kathleen. This is my understanding. If I config initLimit=10 and syncLimit=3, so as follow 2 point: 1. Leader will wait for follow 10*tickTime to do such things: connect to leader, and sync the lasted data from leader. But, if the time follow load snap data from local disk is included in 10*tickTime? A. Load snap data from local disk-->B.Conn to leader-->C. Sync lasted data from leader Maybe, you can tell me, 10*tickTime includes whick part? A or B+C or A+B+C 2. While in runtime, if follow's state too far behind a leadr, even to 3*tickTime, the follow will be dropped. Others, please check my understanding, thx! Charles
> From: [EMAIL PROTECTED] > To: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: RE: About initLimit and syncLimit > Date: Mon, 24 Sep 2012 16:52:14 -0700 > > Thx, kathleen. This is my understanding. If I config initLimit=10 and syncLimit=3, so as follow: 1. Leader will wait for follow 10*tickTime to do such things: connect to leader, and sync the lasted data from leader. But, if the time follow load snap data from local disk is included in 10*tickTime? > > > > > Load snap data from local disk > > > Conn to leader > > > Sync lasted data from leader > > > A: load snap data from local disk,B: conn to leaderC: sync lasted data from leader. Maybe, you can tell me, 10*tickTime includes whick part? A or B+C or A+B+C 2. While in runtime, if follow's state too far behind a leadr, even to 3*tickTime, the follow will be dropped. Others, please check my understanding, thx! Charles > > Date: Mon, 24 Sep 2012 10:19:00 -0500 > > Subject: Re: About initLimit and syncLimit > > From: [EMAIL PROTECTED] > > To: [EMAIL PROTECTED] > > > > Charles, > > > > initLimit measures how much time we anticipate the bootstrapping to > > need whereas syncLimit controls how far behind the Followers can fall > > behind the Leader. Sounds like you might be hitting ZOOKEEPER-1521. As > > a result, can you try increasing both initLimit and syncLimit on all > > ZK servers and then restarting? > > > > Regards, Kathleen > > > > On Mon, Sep 24, 2012 at 8:30 AM, charles charles > > <[EMAIL PROTECTED]> wrote: > > > initLimit > > > Amount of time, in ticks (see tickTime), to allow followers to connect and sync to a leader. Increased this value as needed, if the amount of data managed by ZooKeeper is large. > > > > > > syncLimit > > > Amount of time, in ticks (see tickTime), to allow followers to sync with ZooKeeper. If followers fall too far behind a leader, they will be dropped. In zookeeper config file zoo.cfg, we can config above two parameters, but just from above description, I can't know the difference between initLimit and syncLimit. Anyone can help me, tell me more about them, since my server can't come up, maybe, reson in in these two parameters. > > > > > > Thx! Charles > > > > From: [EMAIL PROTECTED] > > >> To: [EMAIL PROTECTED] > > >> Subject: About initLimit and syncLimit > > >> Date: Mon, 24 Sep 2012 06:25:53 -0700 > > >> > > >> initLimitAmount of time, in ticks (see tickTime), to allow followers to connect and sync to a leader. Increased this value as needed, if the amount of data managed by ZooKeeper is large.syncLimitAmount of time, in ticks (see tickTime), to allow followers to sync with ZooKeeper. If followers fall too far behind a leader, they will be dropped. In zookeeper config file zoo.cfg, we can config above two parameters, but just from above description, I can't know the difference between initLimit and syncLimit. Anyone can help me, tell me more about them, since my server can't come up, maybe, reson in in these two parameters. Thx! Charles > > > > +
charles charles 2012-09-24, 23:59
-
RE: About initLimit and syncLimitramachandra 2012-09-25, 10:48
Hi,
Even I am facing this issue. But I dont want to clean up the data directory, Is there any other way of solving this issue?? Please let me know as soon as possible. Regards, Ramachandra -- View this message in context: http://zookeeper-user.578899.n2.nabble.com/Re-Zookeeper-cluster-failing-tp7577972p7577980.html Sent from the zookeeper-user mailing list archive at Nabble.com. +
ramachandra 2012-09-25, 10:48
-
RE: About initLimit and syncLimitcharles charles 2012-10-04, 09:17
Yeah, I maybe the same problem! Maybe we misunderstand the initLimit and syncLimit.Please check for me: 1. Leader will wait for follow 10*tickTime to do such things: connect to leader, and sync the lasted data from leader. But, if the time follow load snap data from local disk is included in 10*tickTime?
A: load snap data from local disk, B: conn to leader C: sync lasted data from leader. Maybe, you can tell me, 10*tickTime includes which part? A or B+C or A+B+C 2. While in runtime, if follow's state too far behind a leadr, even to 3*tickTime, the follow will be dropped. Others, please check my understanding > Date: Tue, 25 Sep 2012 03:48:47 -0700 > From: [EMAIL PROTECTED] > To: [EMAIL PROTECTED] > Subject: RE: About initLimit and syncLimit > > Hi, > > Even I am facing this issue. But I dont want to clean up the data > directory, Is there any other way of solving this issue?? Please let me know > as soon as possible. > > Regards, > Ramachandra > > > > -- > View this message in context: http://zookeeper-user.578899.n2.nabble.com/Re-Zookeeper-cluster-failing-tp7577972p7577980.html > Sent from the zookeeper-user mailing list archive at Nabble.com. +
charles charles 2012-10-04, 09:17
-
Re: About initLimit and syncLimitPatrick Hunt 2012-10-06, 14:41
Say you have a very large datastore (the snapshot file is very large).
It can take considerable time for the leader to communicate the znode data to the followers during startup. initLimit is the amount of time the leader will allow the followers to d/l the data from the leader and get to a point where they are ready to server requests to clients. If this time is exceeded the leader will close the connection to the follower and the quorum process will restart. 20 sec is typically more than enough time, however if you have a slow connection or the data is very large it might be necessary to increase this. For example I've seen some cases of users storing 100's of gb in their datastore, in which case the initLimit needed to be increased. However in most cases this is not necessary. The syncLimit controls the amount of time the quorum members will allow each other to communicate. If the leader doesn't hear from a follower in the syncLimit time it will drop the follower from the quorum, and vice versa. It's important not to set this parameter too high as it's one of the ways a server detects networking issues and causes recovery to take place (ie the follower will drop out of the quorum and try to reconnect, all clients on that follower will disconnect and reconnect to another server, etc...) Patrick On Thu, Oct 4, 2012 at 2:17 AM, charles charles <[EMAIL PROTECTED]> wrote: > Yeah, I maybe the same problem! Maybe we misunderstand the initLimit and syncLimit.Please check for me: 1. Leader will wait for follow 10*tickTime to do such things: connect to leader, and sync the lasted data from leader. But, if the time follow load snap data from local disk is included in 10*tickTime? > A: load snap data from local disk, > B: conn to leader > C: sync lasted data from leader. > > Maybe, you can tell me, 10*tickTime includes which part? > A or B+C or A+B+C > > 2. While in runtime, if follow's state too far behind a leadr, even to 3*tickTime, the follow will be dropped. > > > Others, please check my understanding > > Date: Tue, 25 Sep 2012 03:48:47 -0700 >> From: [EMAIL PROTECTED] >> To: [EMAIL PROTECTED] >> Subject: RE: About initLimit and syncLimit >> >> Hi, >> >> Even I am facing this issue. But I dont want to clean up the data >> directory, Is there any other way of solving this issue?? Please let me know >> as soon as possible. >> >> Regards, >> Ramachandra >> >> >> >> -- >> View this message in context: http://zookeeper-user.578899.n2.nabble.com/Re-Zookeeper-cluster-failing-tp7577972p7577980.html >> Sent from the zookeeper-user mailing list archive at Nabble.com. > +
Patrick Hunt 2012-10-06, 14:41
-
RE: About initLimit and syncLimitcharles charles 2012-10-09, 11:49
Ok,thank to phunt!
> From: [EMAIL PROTECTED] > Date: Sat, 6 Oct 2012 07:41:16 -0700 > Subject: Re: About initLimit and syncLimit > To: [EMAIL PROTECTED] > CC: [EMAIL PROTECTED] > > Say you have a very large datastore (the snapshot file is very large). > It can take considerable time for the leader to communicate the znode > data to the followers during startup. initLimit is the amount of time > the leader will allow the followers to d/l the data from the leader > and get to a point where they are ready to server requests to clients. > If this time is exceeded the leader will close the connection to the > follower and the quorum process will restart. 20 sec is typically more > than enough time, however if you have a slow connection or the data is > very large it might be necessary to increase this. For example I've > seen some cases of users storing 100's of gb in their datastore, in > which case the initLimit needed to be increased. However in most cases > this is not necessary. > > The syncLimit controls the amount of time the quorum members will > allow each other to communicate. If the leader doesn't hear from a > follower in the syncLimit time it will drop the follower from the > quorum, and vice versa. It's important not to set this parameter too > high as it's one of the ways a server detects networking issues and > causes recovery to take place (ie the follower will drop out of the > quorum and try to reconnect, all clients on that follower will > disconnect and reconnect to another server, etc...) > > Patrick > > On Thu, Oct 4, 2012 at 2:17 AM, charles charles <[EMAIL PROTECTED]> wrote: > > Yeah, I maybe the same problem! Maybe we misunderstand the initLimit and syncLimit.Please check for me: 1. Leader will wait for follow 10*tickTime to do such things: connect to leader, and sync the lasted data from leader. But, if the time follow load snap data from local disk is included in 10*tickTime? > > A: load snap data from local disk, > > B: conn to leader > > C: sync lasted data from leader. > > > > Maybe, you can tell me, 10*tickTime includes which part? > > A or B+C or A+B+C > > > > 2. While in runtime, if follow's state too far behind a leadr, even to 3*tickTime, the follow will be dropped. > > > > > > Others, please check my understanding > > > Date: Tue, 25 Sep 2012 03:48:47 -0700 > >> From: [EMAIL PROTECTED] > >> To: [EMAIL PROTECTED] > >> Subject: RE: About initLimit and syncLimit > >> > >> Hi, > >> > >> Even I am facing this issue. But I dont want to clean up the data > >> directory, Is there any other way of solving this issue?? Please let me know > >> as soon as possible. > >> > >> Regards, > >> Ramachandra > >> > >> > >> > >> -- > >> View this message in context: http://zookeeper-user.578899.n2.nabble.com/Re-Zookeeper-cluster-failing-tp7577972p7577980.html > >> Sent from the zookeeper-user mailing list archive at Nabble.com. > > +
charles charles 2012-10-09, 11:49
|