Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> ASyncProducer loss more messages than SyncProducer when broker crashes.


Copy link to this message
-
Re: ASyncProducer loss more messages than SyncProducer when broker crashes.
We kill the broker on purpose, since we are doing some fail over tests. If
the broker is hard killed, will the async-producer lose message for 6
seconds?

Thanks.

Guodong
On Tue, Feb 5, 2013 at 2:19 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Is the broker shut down cleanly? If so, the producer should get a ZK
> notification immediately. If the broker is hard killed, the ZK notification
> won't arrive at the producer until zk session timeout (default is 6 secs).
>
> Thanks,
>
> Jun
>
> On Mon, Feb 4, 2013 at 7:11 PM, 王国栋 <[EMAIL PROTECTED]> wrote:
>
> > Hi Jun
> >
> > We are using zookeeper. And the async-producer loses  messages in 'X'
> > seconds. Here, X is between 4 and 6. It seems that X is comparable with
> > zookeeper session timeout.
> >
> > I am wondering the behavior of async-producer when socket is broken, but
> > the broker is still visible for zookeeper client.
> >
> >
> > On Tue, Feb 5, 2013 at 12:39 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> >
> > > Currently, there is no callback to report socket failure in the async
> > > producer. However, async producer should be able to detect socket
> failure
> > > and reconnect to another broker. Are you using a vip or Zookeeper in
> the
> > > producer for load balancing?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Sun, Feb 3, 2013 at 10:00 PM, 王国栋 <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hi,
> > > >
> > > > we are doing some fail over test on Kafka0.7.2. We kill one of the
> > > brokers
> > > > on purpose. But we find that the async-producer loss more messages.
> We
> > do
> > > > catch exceptions for send api. But it seems that the async-producer
> > > doesn't
> > > > throw any exceptions when broker is done.
> > > >
> > > > We change the producer to sync-producer, and got the exceptions as
> soon
> > > as
> > > > the broker is down.
> > > >
> > > > So I am wondering if I can get the socket status connected to the
> > broker.
> > > > Does some callbacks can do this?
> > > >
> > > > Thanks.
> > > > --
> > > > Guodong Wang
> > > > 王国栋
> > > >
> > >
> >
> >
> >
> > --
> > Guodong Wang
> > 王国栋
> >
>

--
Guodong Wang
王国栋

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB