Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Log Cleaner in Continuous Loop?


Copy link to this message
-
Re: Log Cleaner in Continuous Loop?
Jun, will KAFKA-292 result in output like this?

$ grep t00-0 log/kafka.log | head
2012-03-25 19:42:03,943 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
2012-03-25 19:43:03,941 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
2012-03-25 19:44:03,942 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
2012-03-25 19:45:03,943 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
2012-03-25 19:46:03,942 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
2012-03-25 19:47:03,943 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
2012-03-25 19:48:03,942 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
2012-03-25 19:49:03,943 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
2012-03-25 19:50:03,941 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
2012-03-25 19:51:03,943 LogManager.scala:201 kafka-logcleaner-0 INFO |
Deleting log segment 00000000000000000000.kafka from t00-0
On 03/24/2012 01:45 AM, Jun Rao wrote:
> https://issues.apache.org/jira/browse/KAFKA-292 can cause this when a topic
> gets new data infrequently. Is that the same problem that you have seen?
>
> Thanks,
>
> Jun
>
> On Fri, Mar 23, 2012 at 6:09 PM, Bateman, Matt <[EMAIL PROTECTED]> wrote:
>
>> Has anyone else see the cleaner get into a continuous loop, attempting to
>> delete the same log files over and over for topic partitions? We've hit
>> this twice now where it never stops until we bounce the broker.
>>
>> I didn't see a JIRA or a mention in the mailing list archives.
>>
>> Thanks,
>>
>> Matt
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB