I am using Kafka 0.8 and message compression can only be enabled on producer side.
However one batch of messages (decided by the producer configuration batch.num.messages) is compressed as a single message and stored at the broker side. I think one batch compressed message will be stored at one broker.
I am wondering in this case how the producer side partitioner class will work? The batch will be partitioned to the partition of the first or last message in the batch? Lex
Re: how is the partitioner working when compression is enabled?
Magnus is right, in both old and new producer, the data is first partitioned, and within each partition batched and compressed for sending.
Guozhang On Tue, Aug 19, 2014 at 7:12 AM, Magnus Edenhill <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation projects and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext