Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Kafka low-latency usage


+
Jakub Ryska 2013-07-24, 15:04
Copy link to this message
-
Re: Kafka low-latency usage
Jay Kreps 2013-07-24, 15:16
Hey Jakub,

Kafka has always been high throughput but has not always been low latency.
In 0.7 and lower the produce request had low latency but the end-to-end
latency (i.e. the time from when the message was sent to when it was
delivered to the consumer) could be in the hundreds of milliseconds or
longer, depending on your configuration.

In 0.8 end-to-end latency should be very low (a few ms) provided that (1)
you disable batch in the producer (enable "sync" mode) and (2) your
consumer can keep up.

-Jay
On Wed, Jul 24, 2013 at 4:07 AM, Jakub Ryska <[EMAIL PROTECTED]>wrote:

> Hi all,
>
> I'm considering to use the Kafka as a low-latency message system. I want
> to have a system, that is able to minimise the delay between sending from
> producer and reading from consumer. I created a test on localhost, I
> created one consumer & producer and I send messages measuring the latency
> an it seems to be very low: ~5 millis.
>
> My question is, it Kafka meant to used as this? One of the use-case that
> Kafka website suggests is activity-stream, that is, a use-case, that is in
> real-time, but if the latency is like 1 second, it is not an issue. My
> use-case is more something like chat room.
>
> The subtitle of Kafka says: "A high-throughput distributed messaging
> system.", does it apply for all the  Kafka versions or did Kafka changed
> over the time? There is a thread in the archive named "end-to-end latency
> in kafka 0.8", It looks like it might be the case, that the Kafka is
> capable/targeting to do low-latency messaging.
>
> Thanks for reading,
> Jakub Ryška
>
>
>
>