Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka snappy compression

Copy link to this message
Re: Kafka snappy compression
Some notes on this:
 - remember that Messages are wrapped in a compressed MessageSet that is
wrapped in an outer Message which is wrapped in an outer MessageSet (the
mind boggles).
 - the Kafka snappy compression has a non-standard framing - snappy-java
(prefix "\x82SNAPPY\x0") - this framing is specific to the java
implementation and is not in the official snappy spec or implementation.

You can use the rdkafka_example client from librdkafka to send
snappy-compressed messages:
./rdkafka_example -P -t topic -p 0 -b mybroker -z snappy

2014-03-27 19:21 GMT+07:00 小宇 <[EMAIL PROTECTED]>: