Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper, mail # user - Maximum size of a snapshot


+
Sergey Maslyakov 2013-07-16, 03:46
+
Thawan Kooburat 2013-07-16, 04:50
+
kishore g 2013-07-16, 07:21
+
Sergey Maslyakov 2013-07-16, 14:16
+
Flavio Junqueira 2013-07-16, 16:16
Copy link to this message
-
Re: Maximum size of a snapshot
kishore g 2013-07-16, 17:04
Thanks for clarification Flavio. Does this mean during the leader election,
both reads and writes are not supported?. Do we start a separate
thread/jira of changing this behavior?.

thanks,
Kishore G
On Tue, Jul 16, 2013 at 9:16 AM, Flavio Junqueira <[EMAIL PROTECTED]>wrote:

> The disk state should be the authoritative state of a server, so if I
> remember correctly, we load the database as a way of validating the disk
> state. I don't claim that this is strictly necessary, but if we are to
> change it, then I would need to think this through.
>
> About leader election, if a leader loses support from a quorum of
> followers,
> then it will drop leadership. Any event that causes a follower to stop
> receiving messages from the leader or the follower to disconnect from the
> leader will make it stop supporting the current leader.
>
> -Flavio
>
> -----Original Message-----
> From: Sergey Maslyakov [mailto:[EMAIL PROTECTED]]
> Sent: 16 July 2013 16:16
> To: [EMAIL PROTECTED]
> Subject: Re: Maximum size of a snapshot
>
> And another extension on top of Kishore's question: do the reelections
> happen if the previously elected leader remains in the cluster? In other
> words, what events can trigger re-election and the corresponding temporary
> degradation of the service provided by Zookeeper?
>
>
> Thank you,
> /Sergey
>
>
> On Tue, Jul 16, 2013 at 2:21 AM, kishore g <[EMAIL PROTECTED]> wrote:
>
> > Regarding #2. Is that really true that during leader election every
> > machine reloads snapshot data from disk? Any reason why this is needed
> > unless it really needs to truncate or undo conflicting transactions
> already applied?
> >
> >
> > On Mon, Jul 15, 2013 at 9:50 PM, Thawan Kooburat <[EMAIL PROTECTED]> wrote:
> >
> > > Max snapshot size:
> > >
> > > Here is my take on these issue,  others feel free to add or correct.
> > >
> > > 1. Depends on how much RAM your machine has.  Snapshot is should be
> > > less than the available RAM since everything is loaded into memory.
> > > 2. Depends on what is the availability guarantee that the client needs.
> > > If there is leader election, every machine need to reload the data
> > > from disk. So the quorum will be down for at least the same as
> > > snapshot
> > loading
> > > time. The session timeout on the client side should be at least
> > > longer than expected downtime during leader election.
> > >
> > > --
> > > Thawan Kooburat
> > >
> > >
> > >
> > >
> > >
> > > On 7/15/13 8:46 PM, "Sergey Maslyakov" <[EMAIL PROTECTED]> wrote:
> > >
> > > >I have a couple of sizing questions to the users and developers.
> > > >Hope,
> > you
> > > >don't mind answering those.
> > > >
> > > >What is the guideline for the maximum reasonable size of a DataTree
> > that a
> > > >single ZK server can manage? If ZK server writes out a snapshot of
> > > >about 1GB in size, is it pushed beyond the limits or is it still
> manageable?
> > If
> > > >so, where is the critical threshold when ZK is really being abused?
> > > >
> > > >Similarly, how can I estimate the propagation delay of a change
> > > >across
> > an
> > > >ensemble of three ZK servers?
> > > >
> > > >
> > > >Thank you,
> > > >/Sergey
> > >
> > >
> >
>
>
+
Flavio Junqueira 2013-07-16, 19:35
+
kishore g 2013-07-16, 20:37
+
Thawan Kooburat 2013-07-16, 22:45
+
kishore g 2013-07-17, 01:46
+
Thawan Kooburat 2013-07-17, 03:40