Kafka, mail # user - Re: Truncated messages in fetch response - 2013-06-26, 16:21
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Truncated messages in fetch response
Yeah, that is true. I thought I documented that, but looking at the
protocol docs, it looks like I didn't.

I agree this is kind of a pain in the ass. It was an important
optimization in 0.7 because we didn't know where the message
boundaries were but in 0.8 we have a fast way to compute message
boundaries and in fact we normally don't give out partial messages, I
think this happens when you hit the size threshold of your fetch
request (e.g. 1MB) instead of searching for the nearest message
boundary we give you that chunk of log. I think we should consider
just fixing it entirely in the next release--the perf hit is pretty
minor and it is an annoyance and source of bugs for clients.

For now you have to handle it, so I added documentation to the protocol wiki.

Sorry!

-Jay

On Wed, Jun 26, 2013 at 8:59 AM, Bob Potter <[EMAIL PROTECTED]> wrote:

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB