Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - Maximum size of a snapshot


Copy link to this message
-
Re: Maximum size of a snapshot
kishore g 2013-07-16, 07:21
Regarding #2. Is that really true that during leader election every machine
reloads snapshot data from disk? Any reason why this is needed unless it
really needs to truncate or undo conflicting transactions already applied?
On Mon, Jul 15, 2013 at 9:50 PM, Thawan Kooburat <[EMAIL PROTECTED]> wrote:

> Max snapshot size:
>
> Here is my take on these issue,  others feel free to add or correct.
>
> 1. Depends on how much RAM your machine has.  Snapshot is should be less
> than the available RAM since everything is loaded into memory.
> 2. Depends on what is the availability guarantee that the client needs.
> If there is leader election, every machine need to reload the data from
> disk. So the quorum will be down for at least the same as snapshot loading
> time. The session timeout on the client side should be at least longer
> than expected downtime during leader election.
>
> --
> Thawan Kooburat
>
>
>
>
>
> On 7/15/13 8:46 PM, "Sergey Maslyakov" <[EMAIL PROTECTED]> wrote:
>
> >I have a couple of sizing questions to the users and developers. Hope, you
> >don't mind answering those.
> >
> >What is the guideline for the maximum reasonable size of a DataTree that a
> >single ZK server can manage? If ZK server writes out a snapshot of about
> >1GB in size, is it pushed beyond the limits or is it still manageable? If
> >so, where is the critical threshold when ZK is really being abused?
> >
> >Similarly, how can I estimate the propagation delay of a change across an
> >ensemble of three ZK servers?
> >
> >
> >Thank you,
> >/Sergey
>
>