Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka snappy compression


Copy link to this message
-
Re: Kafka snappy compression
Some notes on this:
 - remember that Messages are wrapped in a compressed MessageSet that is
wrapped in an outer Message which is wrapped in an outer MessageSet (the
mind boggles).
 - the Kafka snappy compression has a non-standard framing - snappy-java
(prefix "\x82SNAPPY\x0") - this framing is specific to the java
implementation and is not in the official snappy spec or implementation.

You can use the rdkafka_example client from librdkafka to send
snappy-compressed messages:
./rdkafka_example -P -t topic -p 0 -b mybroker -z snappy

2014-03-27 19:21 GMT+07:00 小宇 <[EMAIL PROTECTED]>:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB