Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Broker to consumer compression


Copy link to this message
-
Broker to consumer compression
Pablo Barrera González 2013-04-12, 17:57
Hi

Is it possible to enable compression between the broker and the consumer?

We are thinking in develop this feature in kafka 0.7 but first I would
like to check if there is something out there.

Our escenario is like this:

- the producer is a CPU bounded machine, so we want to keep the CPU
consumption as low as possible, so we can't enable compression here
- the consumers can fetch data from the same data center (no
compression needed) or from a remote data center
- intersite bandwidth is limited so compression would be interesting

Our approach is to add compress the connection at kafka level between
broker and consumer, inside kafka, so the final user can read plain
data.

Regards

Pablo

 
+
Neha Narkhede 2013-04-12, 19:25
+
Pablo Barrera González 2013-04-12, 19:59
+
Neha Narkhede 2013-04-12, 21:32