Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - log.retention.bytes.per.topic does not trigger deletion


Copy link to this message
-
Re: log.retention.bytes.per.topic does not trigger deletion
Jason Rosenberg 2014-01-20, 05:20
Please be sure to update the online config docs with this change!  The
per topic options are still listed there....

Jason

On Thu, Jan 16, 2014 at 9:57 PM, Ben Summer <[EMAIL PROTECTED]> wrote:
> I see. I don't have version 0.8.1 yet. We just updated to 0.8.0 from beta
> after it became the "stable version".
> Good to know there is a fix for this. I'll start trying it out in some
> non-production environments.
>
> Thanks,
> Ben
>
>
> On Thu, Jan 16, 2014 at 7:42 PM, Guozhang Wang <[EMAIL PROTECTED]> wrote:
>
>> In the latest version, per-topic configs have been moved to Zookeeper you
>> and set them using admin tools instead of writing the config files. Could
>> you try trunk HEAD and see if this issue has already been resolved:
>>
>> https://issues.apache.org/jira/browse/KAFKA-554
>>
>> Guozhang
>>
>>
>> On Thu, Jan 16, 2014 at 6:27 PM, Ben Summer <[EMAIL PROTECTED]>
>> wrote:
>>
>> > v0.8.0, (non-beta) downloaded from the website. Let me know if there is a
>> > file I can check in the installation footprint that can give you an
>> actual
>> > change id.
>> >
>> > Is there a fix to this particular feature that you know was made in a
>> later
>> > build?
>> >
>> > Thanks for the quick response,
>> > Ben
>> >
>> >
>> > On Thu, Jan 16, 2014 at 6:22 PM, Guozhang Wang <[EMAIL PROTECTED]>
>> wrote:
>> >
>> > > Hello Ben,
>> > >
>> > > Which version are you using?
>> > >
>> > > Guozhang
>> > >
>> > >
>> > > On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer <[EMAIL PROTECTED]>
>> > > wrote:
>> > >
>> > > > I tried using the following two retention properties
>> > > >
>> > > > log.retention.bytes=3221225472
>> > > >
>> > > >
>> > >
>> >
>> log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776
>> > > >
>> > > > which I interpret to mean "by default, keep 3GB per topic partition,
>> > > except
>> > > > for first_topic and second_topic, which should retain 1TB each
>> (across
>> > > all
>> > > > their partitions).
>> > > >
>> > > > I've noticed that the data for first_topic and second_topic grow way
>> > > beyond
>> > > > 1TB and no deletions occur. Is the syntax correct?
>> > > >
>> > > > Thanks in advance,
>> > > > Ben
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > -- Guozhang
>> > >
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>