Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> ASyncProducer loss more messages than SyncProducer when broker crashes.


Copy link to this message
-
Re: ASyncProducer loss more messages than SyncProducer when broker crashes.
Hi Jun

We are using zookeeper. And the async-producer loses  messages in 'X'
seconds. Here, X is between 4 and 6. It seems that X is comparable with
zookeeper session timeout.

I am wondering the behavior of async-producer when socket is broken, but
the broker is still visible for zookeeper client.
On Tue, Feb 5, 2013 at 12:39 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Currently, there is no callback to report socket failure in the async
> producer. However, async producer should be able to detect socket failure
> and reconnect to another broker. Are you using a vip or Zookeeper in the
> producer for load balancing?
>
> Thanks,
>
> Jun
>
> On Sun, Feb 3, 2013 at 10:00 PM, 王国栋 <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > we are doing some fail over test on Kafka0.7.2. We kill one of the
> brokers
> > on purpose. But we find that the async-producer loss more messages. We do
> > catch exceptions for send api. But it seems that the async-producer
> doesn't
> > throw any exceptions when broker is done.
> >
> > We change the producer to sync-producer, and got the exceptions as soon
> as
> > the broker is down.
> >
> > So I am wondering if I can get the socket status connected to the broker.
> > Does some callbacks can do this?
> >
> > Thanks.
> > --
> > Guodong Wang
> > 王国栋
> >
>

--
Guodong Wang
王国栋

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB