Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> questtion about log.retention.bytes


Copy link to this message
-
Re: questtion about log.retention.bytes
log.retention.bytes is for all topics that are not included in
log.retention.bytes.per.topic
(which defines a map of topic -> size).

Currently, we don't have a total size limit across all topics.

Thanks,

Jun
On Fri, Aug 16, 2013 at 2:00 PM, Paul Christian
<[EMAIL PROTECTED]>wrote:

> According to the Kafka 8 documentation under broker configuration. There
> are these parameters and their definitions.
>
> log.retention.bytes -1 The maximum size of the log before deleting it
> log.retention.bytes.per.topic "" The maximum size of the log for some
> specific topic before deleting it
>
> I'm curious what the first value 'log.retention.bytes' is for if the second
> one is for per topic logs, because aren't all logs generated per topic? Is
> this an aggregate value across topics?
>
> Related question, is there a parameter for kafka where you can say only
> hold this much TOTAL data across all topic ( logs/index together )? I.e.
> our  hosts have this much available space and so value
> log.retention.whatever.aggregate == 75% total  disk space.
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB