Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Producer / Consumer - connection management


Copy link to this message
-
Re: Producer / Consumer - connection management
> Basically, who and how is managing network disconnection / outage ?

Short answer is that the producer is responsible for retrying connection
establishment and it automatically does so when the network connection to
the broker fails. There is a connection retry interval that you can use.
Once that is reached it gives up trying to reconnect.

> 1 ) If the "sync" producer has been created successfully, will it
reconnect whenever it loses its connection to the broker or zk ? I could
understand it's not happening as we expect the connection to be immediately
available when using the sync producer.

The ZK client library manages connection loss and reconnects to ZK. But be
aware that ZK not being unavailable is never an option.

> 2 ) How about the "async" producer ? Does it expect connection at
creation time ? will it reconnect in case of failure ?

Can you describe what connection issue you see at creation time. Is it just
when you use the zk.connect option ?

> 3 ) Finally, how about the "high level" consumer ? Afaik, it reconnects
automatically

It does, so does producer.

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB