I have created one node kafka cluster and one topic name test1. Then I started ConsoleProducer and ConsoleConsumer.Both started successfully.But what I see is in the /kafka/kafka_2.8.0-0.8.0/logs/ directory
-rw-rw-r-- 1 admin admin 5203 Jan 23 15:37 controller.log *-rw-rw-r-- 1 admin admin 2589466 Jan 23 15:45 kafka-request.log* -rw-rw-r-- 1 admin admin 2604 Jan 23 15:44 kafkaServer-gc.log -rw-rw-r-- 1 admin admin 6505 Jan 23 15:40 server.log -rw-rw-r-- 1 admin admin 2167 Jan 23 15:37 state-change.log
The size of file *kafka-request.log* is continuously increasing even though I am not giving any message to producer. When I killed ConsoleConsumer and ConsoleProducer,the size of file *kafka-request.log *stopped increasing. So I want to know the reason why this is(*kafka-request.log) *is only file whose size is getting increased continuously even though no message is given to produce or no message is consumed by consumer. *Moniii*
Well, the consumer keeps sending fetch requests even when there are no more messages. Even fetch request will only block on the broker for up to max.wait.time. Also, the producer periodically issues metadata requests, which are also logged in kafka-request.log.
Jun On Thu, Jan 23, 2014 at 2:20 AM, Monika Garg <[EMAIL PROTECTED]> wrote:
It means my free disk size will keep on reducing till Producer and Consumer will be running?Is there any way to stop the increment in file size (kafka-request.log)? On Thu, Jan 23, 2014 at 8:37 PM, Jun Rao <[EMAIL PROTECTED]> wrote: *Moniii*
The producer/consumer log compaction/retention can be set in log4j.
Guozhang On Thu, Jan 23, 2014 at 7:57 AM, Monika Garg <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext