Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> broker not release segment file after deleting expired segments?


Copy link to this message
-
Re: broker not release segment file after deleting expired segments?
The mailing list does not accept attachments - you will need to either
paste here or externally, say on gist.

Thanks,

Joel

On Wed, Aug 8, 2012 at 2:52 PM, xiaoyu wang <[EMAIL PROTECTED]> wrote:

> Neha,
>
> I have attached the du and ls output for one of the partitions. Looks like
> for some segments, du is showing 1GB while ls shows 513MB. Let me know if
> you need additional info.
>
> Thanks,
>
> -Xiaoyu
>
>
>
> On Wed, Aug 8, 2012 at 12:29 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
>
>> Xiaoyu,
>>
>> Please could you paste the log statements that specify which files are
>> deleted by the broker and also the detailed output by du on the Kafka
>> log directory ?
>> If the broker says it has garbage collected some log segments, space
>> should free up.
>>
>> Thanks,
>> Neha
>>
>> On Wed, Aug 8, 2012 at 12:22 PM, xiaoyu wang <[EMAIL PROTECTED]>
>> wrote:
>> > Hi all,
>> >
>> > I am running kafka-0.7.0 and found that even though expired segments are
>> > deleted by broker, the disk space is not released. So, du shows that a
>> > partition is 450 GB, but the total size of the files in the partition is
>> > only about 250GB. Is this a bug? We are using raid10.
>> >
>> > Thanks,
>> >
>> > -Xiaoyu
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB