Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> message ordering guarantees


Copy link to this message
-
Re: message ordering guarantees
In 0.7, each producer reconnects to the broker periodically. Messages sent
in different socket connections could arrive out of order. Take a look at
reconnect.interval in http://kafka.apache.org/07/configuration.html. If you
don't use VIP for the producers, you can actually set reconnect.interval to
a really large value. In 0.8, there is no reconnect.interval any more.
Messages sent from a single producer are expected to arrive in order at the
broker, as long as they sent to the same partition in a topic.

Thanks,

Jun
On Tue, May 21, 2013 at 5:42 PM, Ross Black <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am using Kafka 0.7.1, and using SyncProducer and SimpleConsumer with a
> single broker service process.
>
> I am occasionally seeing messages (from a *single* partition) being
> processed out of order to what I expect and I am trying to find where the
> problem lies.  The problem may well be in my code - I just would like to
> eliminate Kafka as a potential cause.
>
> Messages are being sent sequentially from the producer process, using a
> single SyncProducer.
> Does Kafka provide any guarantees for message ordering in this case?
>
> eg.  If The sync producer sends messages A then B then C, does the Kafka
> broker guarantee that messages will be persisted with the order A,B,C?
> If not, is there any way to ensure this ordering?
> Has anything changed in 0.8 that could be used to ensure that ordering?
>
>
> Thanks,
> Ross
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB