Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> Changed ordering guarantee with multiple in-flight messages


Copy link to this message
-
Re: Changed ordering guarantee with multiple in-flight messages
I can elaborate further on the wiki tomorrow. The term in-flight in my
edit is a bit incomplete. It refers to what's "in-flight" on the
broker-side for actual handling - that is what provides the ordering
guarantee. The client can continue to write requests to the socket
even while the broker is handling a preceding request since those
requests will sit in the socket buffer. So there is still significant
benefit in using non-blocking IO/request pipelining on the client-side
to achieve high throughput.

Thanks,

Joel
On Thu, Nov 21, 2013 at 11:33 PM, Magnus Edenhill <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I noticed Joel Koshy's update to the protocol guide wiki at
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol
>
> This sentence was added:
> "The broker allows only a single in-flight request per connection in order
> to guarantee this ordering"
>
> Adding such a constraint for the number of in-flight requests is a
> performance killer, and it seems odd to do so at this point in time given
> the number of third-party clients implemented - at least some of them
> hopefully relying on multiple requests in flight being properly supported.
>
> It is also in contrast with the previous sentence:
> "The server guarantees that on a single TCP connection, requests will be
> processed in the order they are sent and responses will return in that
> order as well."
>
>
> So, can you elaborate on this new constraint?
> In which cases with multiple in-flight requests may reordering reoccur?
>
> Regards,
> Magnus

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB