Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Message latency

Copy link to this message
Re: Message latency
Jay Kreps 2014-02-04, 21:27
There are two definitions of latency:
1. How long before the writer gets an acknowledgement for their write. This
depends on the acks setting the producer has as Guozhang says. If acks = 1
we wait just on the leader, if acks=-1 we wait on all "in sync" brokers
(i.e. alive brokers).
2. How long before the consumer gets the message. This always blocks on
full acknowledgement by "in sync" brokers.

So both can be dependent on the latency of the followers, however if a
followers falls behind it will be removed from the in sync set, so there is
a bound.

On Tue, Feb 4, 2014 at 11:57 AM, Michael Popov <[EMAIL PROTECTED]> wrote:

> Hi,
> I am trying to understand what influences message latency in Kafka.
> According to the Kafka Replication Design document (
> https://linkedin.jira.com/secure/attachment/10050/kafka_replication_highlevel_design.pdf
> ):
> 1.       Writes: Once the leader receives the acknowledgment from all
> replicas in ISR, the message is committed. The leader advances the HW and
> sends an acknowledgment to the client (page 4)
> 2.       Reads: Only messages up to the HW are exposed to the reader (page
> 5)
> It means message latency depends on how fast followers pulled data from
> the leader and reported successful data writes.
> Is this correct?
> Thank you,
> Michael Popov