Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - message ordering guarantees


Copy link to this message
-
Re: message ordering guarantees
Jun Rao 2013-05-22, 04:19
In 0.7, each producer reconnects to the broker periodically. Messages sent
in different socket connections could arrive out of order. Take a look at
reconnect.interval in http://kafka.apache.org/07/configuration.html. If you
don't use VIP for the producers, you can actually set reconnect.interval to
a really large value. In 0.8, there is no reconnect.interval any more.
Messages sent from a single producer are expected to arrive in order at the
broker, as long as they sent to the same partition in a topic.

Thanks,

Jun
On Tue, May 21, 2013 at 5:42 PM, Ross Black <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am using Kafka 0.7.1, and using SyncProducer and SimpleConsumer with a
> single broker service process.
>
> I am occasionally seeing messages (from a *single* partition) being
> processed out of order to what I expect and I am trying to find where the
> problem lies.  The problem may well be in my code - I just would like to
> eliminate Kafka as a potential cause.
>
> Messages are being sent sequentially from the producer process, using a
> single SyncProducer.
> Does Kafka provide any guarantees for message ordering in this case?
>
> eg.  If The sync producer sends messages A then B then C, does the Kafka
> broker guarantee that messages will be persisted with the order A,B,C?
> If not, is there any way to ensure this ordering?
> Has anything changed in 0.8 that could be used to ensure that ordering?
>
>
> Thanks,
> Ross
>