Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Questions on snapshots

Raghu Shastry 2013-02-14, 03:09
Copy link to this message
Re: Questions on snapshots
The logic that controls rolling the logs and taking a snap is:
randRoll = random next in between 0 and snapCount/2
if we've logged  more than snapCount/2 + randRoll, create a new randRoll
and take a snapshot.

So creating a snapshot after ~72K transactions indicates to me that you
probably got the default 100K for the snapCount. Not sure why it didn't get
read, though, ZooKeeperServer does a
System.getProperty("zookeeper.snapCount") and parses that int for the
snapCount, defaulting to 100000 if it fails. Possibly just a formatting
issue in your configs?
Yes, if you don't snapshot regularly, the transaction log will keep growing
in size and only get rolled when it is time to do a snapshot.

On Wed, Feb 13, 2013 at 10:09 PM, Raghu Shastry <[EMAIL PROTECTED]> wrote:

> I am returning to the ZooKeeper land after a long hiatus, so got
> some questions :-)
> I am running some tests trying to reduce the frequency of ZK snapshots. So
> I am setting zookeeper.snapCount to a very high value (10
> million transactions) so that it forces ZK to create snapshots
> infrequently. ZK seems to be ignoring this completely and creating
> snapshots once about 72K transactions (I thought the default snapCount
> value is 100K). I have the following line in the config file:
> zookeeper.snapCount=10000000
> Couple of questions:
> 1. Any idea why ZK is not honoring the snapCount value? Any idea why it is
> not honoring the default value either?
> 2. If I understand it correctly, the transaction log is reset each time
> (and only when) a snapshot is taken. If snapshots are not taken regularly,
> the transaction log will keep growing in size without ever being truncated
> to shrink it's size. Did I describe this right?
> Thanks
> Raghu