Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> log.retention.bytes.per.topic does not trigger deletion


Copy link to this message
-
Re: log.retention.bytes.per.topic does not trigger deletion
Please be sure to update the online config docs with this change!  The
per topic options are still listed there....

Jason

On Thu, Jan 16, 2014 at 9:57 PM, Ben Summer <[EMAIL PROTECTED]> wrote:
> I see. I don't have version 0.8.1 yet. We just updated to 0.8.0 from beta
> after it became the "stable version".
> Good to know there is a fix for this. I'll start trying it out in some
> non-production environments.
>
> Thanks,
> Ben
>
>
> On Thu, Jan 16, 2014 at 7:42 PM, Guozhang Wang <[EMAIL PROTECTED]> wrote:
>
>> In the latest version, per-topic configs have been moved to Zookeeper you
>> and set them using admin tools instead of writing the config files. Could
>> you try trunk HEAD and see if this issue has already been resolved:
>>
>> https://issues.apache.org/jira/browse/KAFKA-554
>>
>> Guozhang
>>
>>
>> On Thu, Jan 16, 2014 at 6:27 PM, Ben Summer <[EMAIL PROTECTED]>
>> wrote:
>>
>> > v0.8.0, (non-beta) downloaded from the website. Let me know if there is a
>> > file I can check in the installation footprint that can give you an
>> actual
>> > change id.
>> >
>> > Is there a fix to this particular feature that you know was made in a
>> later
>> > build?
>> >
>> > Thanks for the quick response,
>> > Ben
>> >
>> >
>> > On Thu, Jan 16, 2014 at 6:22 PM, Guozhang Wang <[EMAIL PROTECTED]>
>> wrote:
>> >
>> > > Hello Ben,
>> > >
>> > > Which version are you using?
>> > >
>> > > Guozhang
>> > >
>> > >
>> > > On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer <[EMAIL PROTECTED]>
>> > > wrote:
>> > >
>> > > > I tried using the following two retention properties
>> > > >
>> > > > log.retention.bytes=3221225472
>> > > >
>> > > >
>> > >
>> >
>> log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776
>> > > >
>> > > > which I interpret to mean "by default, keep 3GB per topic partition,
>> > > except
>> > > > for first_topic and second_topic, which should retain 1TB each
>> (across
>> > > all
>> > > > their partitions).
>> > > >
>> > > > I've noticed that the data for first_topic and second_topic grow way
>> > > beyond
>> > > > 1TB and no deletions occur. Is the syntax correct?
>> > > >
>> > > > Thanks in advance,
>> > > > Ben
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > -- Guozhang
>> > >
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB