Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> how is the partitioner working when compression is enabled?


Copy link to this message
-
how is the partitioner working when compression is enabled?
I am using Kafka 0.8 and message compression can only be enabled on producer side.

However one batch of messages (decided by the producer configuration batch.num.messages) is compressed as a single message and stored at the broker side. I think one batch compressed message will be stored at one broker.

I am wondering in this case how the producer side partitioner class will work? The batch will be partitioned to the partition of the first or last message in the batch?
Lex