Home | About | Sematext search-lucene.com search-hadoop.com search-devops.com
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> Troubles with compressed message set


+
David Arthur 2013-01-30, 15:27
+
Jay Kreps 2013-01-30, 16:19
+
David Arthur 2013-01-30, 21:00
Copy link to this message
-
Re: Troubles with compressed message set

On 1/30/13 11:18 AM, Jay Kreps wrote:
Other repeated elements in the protocol are preceded by a int32 to
indicate the number of repeated elements. E.g., topicCount in
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/api/ProducerRequest.scala#L38

MessageSet does not do this, it just starts writing out the messages
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/message/ByteBufferMessageSet.scala#L35.
To be more like the rest of the protocol, it would need to first do
something like: buffer.putInt(messages.size)

It's not really a big deal either way, but I thought it important to
point out since it's different than the rest of the protocol and it
tripped me up.

-David
 
+
Jay Kreps 2013-01-30, 21:17
+
David Arthur 2013-01-30, 21:23
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB