Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> ASyncProducer loss more messages than SyncProducer when broker crashes.


+
王国栋 2013-02-04, 06:01
+
Jun Rao 2013-02-04, 16:44
Copy link to this message
-
Re: ASyncProducer loss more messages than SyncProducer when broker crashes.
Hi Jun

We are using zookeeper. And the async-producer loses  messages in 'X'
seconds. Here, X is between 4 and 6. It seems that X is comparable with
zookeeper session timeout.

I am wondering the behavior of async-producer when socket is broken, but
the broker is still visible for zookeeper client.
On Tue, Feb 5, 2013 at 12:39 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Currently, there is no callback to report socket failure in the async
> producer. However, async producer should be able to detect socket failure
> and reconnect to another broker. Are you using a vip or Zookeeper in the
> producer for load balancing?
>
> Thanks,
>
> Jun
>
> On Sun, Feb 3, 2013 at 10:00 PM, 王国栋 <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > we are doing some fail over test on Kafka0.7.2. We kill one of the
> brokers
> > on purpose. But we find that the async-producer loss more messages. We do
> > catch exceptions for send api. But it seems that the async-producer
> doesn't
> > throw any exceptions when broker is done.
> >
> > We change the producer to sync-producer, and got the exceptions as soon
> as
> > the broker is down.
> >
> > So I am wondering if I can get the socket status connected to the broker.
> > Does some callbacks can do this?
> >
> > Thanks.
> > --
> > Guodong Wang
> > 王国栋
> >
>

--
Guodong Wang
王国栋

 
+
Jun Rao 2013-02-05, 06:20
+
王国栋 2013-02-05, 06:31
+
Jun Rao 2013-02-05, 09:03
+
王国栋 2013-02-05, 12:55