Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - ASyncProducer loss more messages than SyncProducer when broker crashes.


Copy link to this message
-
Re: ASyncProducer loss more messages than SyncProducer when broker crashes.
Jun Rao 2013-02-05, 06:20
Is the broker shut down cleanly? If so, the producer should get a ZK
notification immediately. If the broker is hard killed, the ZK notification
won't arrive at the producer until zk session timeout (default is 6 secs).

Thanks,

Jun

On Mon, Feb 4, 2013 at 7:11 PM, 王国栋 <[EMAIL PROTECTED]> wrote:

> Hi Jun
>
> We are using zookeeper. And the async-producer loses  messages in 'X'
> seconds. Here, X is between 4 and 6. It seems that X is comparable with
> zookeeper session timeout.
>
> I am wondering the behavior of async-producer when socket is broken, but
> the broker is still visible for zookeeper client.
>
>
> On Tue, Feb 5, 2013 at 12:39 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Currently, there is no callback to report socket failure in the async
> > producer. However, async producer should be able to detect socket failure
> > and reconnect to another broker. Are you using a vip or Zookeeper in the
> > producer for load balancing?
> >
> > Thanks,
> >
> > Jun
> >
> > On Sun, Feb 3, 2013 at 10:00 PM, 王国栋 <[EMAIL PROTECTED]> wrote:
> >
> > > Hi,
> > >
> > > we are doing some fail over test on Kafka0.7.2. We kill one of the
> > brokers
> > > on purpose. But we find that the async-producer loss more messages. We
> do
> > > catch exceptions for send api. But it seems that the async-producer
> > doesn't
> > > throw any exceptions when broker is done.
> > >
> > > We change the producer to sync-producer, and got the exceptions as soon
> > as
> > > the broker is down.
> > >
> > > So I am wondering if I can get the socket status connected to the
> broker.
> > > Does some callbacks can do this?
> > >
> > > Thanks.
> > > --
> > > Guodong Wang
> > > 王国栋
> > >
> >
>
>
>
> --
> Guodong Wang
> 王国栋
>