Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - broker not release segment file after deleting expired segments?


Copy link to this message
-
Re: broker not release segment file after deleting expired segments?
Joel Koshy 2012-08-08, 21:58
The mailing list does not accept attachments - you will need to either
paste here or externally, say on gist.

Thanks,

Joel

On Wed, Aug 8, 2012 at 2:52 PM, xiaoyu wang <[EMAIL PROTECTED]> wrote:

> Neha,
>
> I have attached the du and ls output for one of the partitions. Looks like
> for some segments, du is showing 1GB while ls shows 513MB. Let me know if
> you need additional info.
>
> Thanks,
>
> -Xiaoyu
>
>
>
> On Wed, Aug 8, 2012 at 12:29 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
>
>> Xiaoyu,
>>
>> Please could you paste the log statements that specify which files are
>> deleted by the broker and also the detailed output by du on the Kafka
>> log directory ?
>> If the broker says it has garbage collected some log segments, space
>> should free up.
>>
>> Thanks,
>> Neha
>>
>> On Wed, Aug 8, 2012 at 12:22 PM, xiaoyu wang <[EMAIL PROTECTED]>
>> wrote:
>> > Hi all,
>> >
>> > I am running kafka-0.7.0 and found that even though expired segments are
>> > deleted by broker, the disk space is not released. So, du shows that a
>> > partition is 450 GB, but the total size of the files in the partition is
>> > only about 250GB. Is this a bug? We are using raid10.
>> >
>> > Thanks,
>> >
>> > -Xiaoyu
>>
>
>