Kafka, mail # user - Re: Truncated messages in fetch response - 2013-06-26, 16:21
 Search Hadoop and all its subprojects:

Switch to Plain View
Bob Potter 2013-06-26, 16:00
Copy link to this message
Re: Truncated messages in fetch response
Yeah, that is true. I thought I documented that, but looking at the
protocol docs, it looks like I didn't.

I agree this is kind of a pain in the ass. It was an important
optimization in 0.7 because we didn't know where the message
boundaries were but in 0.8 we have a fast way to compute message
boundaries and in fact we normally don't give out partial messages, I
think this happens when you hit the size threshold of your fetch
request (e.g. 1MB) instead of searching for the nearest message
boundary we give you that chunk of log. I think we should consider
just fixing it entirely in the next release--the perf hit is pretty
minor and it is an annoyance and source of bugs for clients.

For now you have to handle it, so I added documentation to the protocol wiki.



On Wed, Jun 26, 2013 at 8:59 AM, Bob Potter <[EMAIL PROTECTED]> wrote:

Vadim Keylis 2013-06-27, 19:30
Bob Potter 2013-06-27, 22:30
Milind Parikh 2013-06-27, 22:53
Jay Kreps 2013-06-27, 23:06
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB