Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> Changed ordering guarantee with multiple in-flight messages


Copy link to this message
-
Re: Changed ordering guarantee with multiple in-flight messages
I can elaborate further on the wiki tomorrow. The term in-flight in my
edit is a bit incomplete. It refers to what's "in-flight" on the
broker-side for actual handling - that is what provides the ordering
guarantee. The client can continue to write requests to the socket
even while the broker is handling a preceding request since those
requests will sit in the socket buffer. So there is still significant
benefit in using non-blocking IO/request pipelining on the client-side
to achieve high throughput.

Thanks,

Joel
On Thu, Nov 21, 2013 at 11:33 PM, Magnus Edenhill <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I noticed Joel Koshy's update to the protocol guide wiki at
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol
>
> This sentence was added:
> "The broker allows only a single in-flight request per connection in order
> to guarantee this ordering"
>
> Adding such a constraint for the number of in-flight requests is a
> performance killer, and it seems odd to do so at this point in time given
> the number of third-party clients implemented - at least some of them
> hopefully relying on multiple requests in flight being properly supported.
>
> It is also in contrast with the previous sentence:
> "The server guarantees that on a single TCP connection, requests will be
> processed in the order they are sent and responses will return in that
> order as well."
>
>
> So, can you elaborate on this new constraint?
> In which cases with multiple in-flight requests may reordering reoccur?
>
> Regards,
> Magnus