Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - auto-purge not purging


Copy link to this message
-
Re: auto-purge not purging
Ryan Fox 2013-11-04, 15:46
>
> What version of ZooKeeper are you running? (solr 4.1.0 is pretty old,
> 4.5.1 is out now, perhaps try upgrading?)
>
> Zookeeper version is 3.4.5.  This is part of a large software package, so
upgrading the solr library would be painful, but possible.
"do not get autopurged" - could you provide more insight. 1) are you
> sure the configuration options are being applied? 2) what does the
> datadir look like? (snaps/logs, perhaps do an "ls -la" and provide in
> this thread) 3) anything autopurge related in the zk server logs?
>
>  Sure, the datadir contains only logs and snapshots.  I would expect the
autoPurge function to delete logs and snapshots older than the number I
have set in zoo.cfg, but nothing gets deleted, and they just pile up,
eating up my hard drive.
1) The zookeeper instance is picking up my settings for clientPort, so I
assume I have the purge settings in the correct place as well.  The
relevant section of zoo.cfg:
autopurge.snapRetainCount=3
autopurge.purgeInterval=1

2) ls -lah
-rw-r--r--  1 ryanfox  staff    64M Oct 30 13:09 log.1
-rw-r--r--  1 ryanfox  staff    64M Nov  4 09:36 log.48
-rw-r--r--  1 ryanfox  staff    64M Nov  4 09:37 log.7b
-rw-r--r--  1 ryanfox  staff   296B Oct 30 12:53 snapshot.0
-rw-r--r--  1 ryanfox  staff    60K Nov  4 09:34 snapshot.47
-rw-r--r--  1 ryanfox  staff    60K Nov  4 09:36 snapshot.7a

A new log and snapshot gets created every time I stop and start solr.  As
an aside, every log file appears to be 64MB, no matter how much/little is
in my index.  Is that correct?  Seems like a lot for an empty solr instance.

3) I haven't seen any smoking gun in the logs, but haven't specifically
looked, either.  I'll let it go until it should have triggered the
auto-purge and report back.

Thanks for your help in this.

Ryan