-Re: Broker to consumer compression
Pablo Barrera González 2013-04-12, 19:59
Thanks for the replay Neha, but that's is end-to-end and I am looking
for a broker-consumer compression.
Producer -> uncompressed -> broker -> compressed -> consumer
2013/4/12 Neha Narkhede <[EMAIL PROTECTED]>:
> Kafka already supports end-to-end compression which means data
> transfer between brokers and consumers is compressed. There are two
> supported compression codecs - GZIP and Snappy. The latter is lighter
> on CPU consumption. See this blog post for comparison -
> On Fri, Apr 12, 2013 at 10:56 AM, Pablo Barrera González
> <[EMAIL PROTECTED]> wrote:
>> Is it possible to enable compression between the broker and the consumer?
>> We are thinking in develop this feature in kafka 0.7 but first I would
>> like to check if there is something out there.
>> Our escenario is like this:
>> - the producer is a CPU bounded machine, so we want to keep the CPU
>> consumption as low as possible, so we can't enable compression here
>> - the consumers can fetch data from the same data center (no
>> compression needed) or from a remote data center
>> - intersite bandwidth is limited so compression would be interesting
>> Our approach is to add compress the connection at kafka level between
>> broker and consumer, inside kafka, so the final user can read plain