We have 3 brokers in a cluster. The replication factor is 2. I set the default retention size to 3G bytes. I published 12G data to a topic, which is enough to fully load all partitions. I assume on each broker the partition size should be 3G. However, it is only 1.4G for one partation. Is this a known issue? Thanks.
Are you on 0.8 branch or trunk? Also, what are the sizes for the other partitions? Also how many producers do you have? It could be because the producer sticks to a partition for the metadata refresh period - so if your test run isn't long enough some partitions may be more loaded than the others.
On Thu, Nov 21, 2013 at 06:28:39PM +0000, Yu, Libo wrote:
I did a restart and the issue was gone. It could be that we changed the retention size and did not restart the brokers to pick up the change. Thanks for your help.
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation projects and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext