Kafka, mail # user - Re: who is using kafka to stare large messages? - 2013-10-08, 16:37
 Search Hadoop and all its subprojects:

Switch to Plain View
MB JA 2014-05-17, 03:57
S Ahmed 2013-10-07, 14:45
Jun Rao 2013-10-07, 15:55
S Ahmed 2013-10-07, 16:10
Neha Narkhede 2013-10-07, 16:22
S Ahmed 2013-10-07, 16:27
Benjamin Black 2013-10-07, 16:37
Neha Narkhede 2013-10-07, 16:41
Jason Rosenberg 2013-10-08, 06:36
Neha Narkhede 2013-10-08, 14:20
Jason Rosenberg 2013-10-08, 16:25
Copy link to this message
Re: who is using kafka to stare large messages?
And to be clear, if uncompressed messages come in, they remain uncompressed
in the broker, correct?


Currently, only the broker has knowledge of the offsets for a partition and
hence is the right place to assign the offsets. Even if the producer sends
metadata, the broker still needs to decompress the data in order to get a
handle to the individual message in order to assign the logical offset.

One of the JIRAs discussing this is here -


On Tue, Oct 8, 2013 at 9:24 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB