Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> auto-purge not purging


Copy link to this message
-
Re: auto-purge not purging
>
> What version of ZooKeeper are you running? (solr 4.1.0 is pretty old,
> 4.5.1 is out now, perhaps try upgrading?)
>
> Zookeeper version is 3.4.5.  This is part of a large software package, so
upgrading the solr library would be painful, but possible.
"do not get autopurged" - could you provide more insight. 1) are you
> sure the configuration options are being applied? 2) what does the
> datadir look like? (snaps/logs, perhaps do an "ls -la" and provide in
> this thread) 3) anything autopurge related in the zk server logs?
>
>  Sure, the datadir contains only logs and snapshots.  I would expect the
autoPurge function to delete logs and snapshots older than the number I
have set in zoo.cfg, but nothing gets deleted, and they just pile up,
eating up my hard drive.
1) The zookeeper instance is picking up my settings for clientPort, so I
assume I have the purge settings in the correct place as well.  The
relevant section of zoo.cfg:
autopurge.snapRetainCount=3
autopurge.purgeInterval=1

2) ls -lah
-rw-r--r--  1 ryanfox  staff    64M Oct 30 13:09 log.1
-rw-r--r--  1 ryanfox  staff    64M Nov  4 09:36 log.48
-rw-r--r--  1 ryanfox  staff    64M Nov  4 09:37 log.7b
-rw-r--r--  1 ryanfox  staff   296B Oct 30 12:53 snapshot.0
-rw-r--r--  1 ryanfox  staff    60K Nov  4 09:34 snapshot.47
-rw-r--r--  1 ryanfox  staff    60K Nov  4 09:36 snapshot.7a

A new log and snapshot gets created every time I stop and start solr.  As
an aside, every log file appears to be 64MB, no matter how much/little is
in my index.  Is that correct?  Seems like a lot for an empty solr instance.

3) I haven't seen any smoking gun in the logs, but haven't specifically
looked, either.  I'll let it go until it should have triggered the
auto-purge and report back.

Thanks for your help in this.

Ryan
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB