Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Why do we need reconnect with an interval?


Copy link to this message
-
Re: Why do we need reconnect with an interval?
If use you zk.connect in the producer, you can leave reconnect.interval as
maxInt since we do load balancing using ZK. If you use a single VIP in
broker.list in the producer,  reconnect.interval will help load balancing.

Thanks,

Jun

On Sun, Nov 18, 2012 at 10:15 PM, Bae, Jae Hyeon <[EMAIL PROTECTED]> wrote:

> Thank you for your answer.
>
> SyncProducer doesn't change its host and port when reconnecting. So, I
> don't understand why reconnection can be helpful for load balancing.
> When the new broker is registered in the zookeeper, it will be added
> in the ProducerPool by producerCbk, which does not seem related with
> balancing.
>
> So, is it totally safe to change reconnect.interval as
> Integer.MAX_VALUE and reconnect.time.interval.ms as -1 in the
> environment where the broker can be newly created?
>
> On Sun, Nov 18, 2012 at 9:57 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > In addition to that, reconnect interval is useful for load balancing. If
> a
> > broker is restarted, initially, no existing producers are connected to
> it.
> > Reconnect interval gives the producer a chance to connect to the new
> broker
> > periodically.
> >
> > Thanks,
> >
> > Jun
> >
> > On Sun, Nov 18, 2012 at 3:47 PM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
> >
> >> Hi,
> >>
> >> This config is really relevant to 0.7 Kafka users using a VIP to talk
> >> to the Kafka cluster. VIP has a timeout for every TCP connection,
> >> where it cuts the connection that has been idle for a certain time.
> >> Most VIPs are configured to not send TCP RST back to the client, so
> >> the client does not know that the tcp connection is no more. The next
> >> time the client tries to write to the socket, it gets into a really
> >> long TCP RST retry loop. To avoid this, we added a producer config
> >> that would reconnect to the VIP after n requests or m ms, whatever
> >> comes earlier.
> >>
> >> This config is no longer required in 0.8 and has been removed.
> >>
> >> Thanks,
> >> Neha
> >>
> >> On Sun, Nov 18, 2012 at 3:37 PM, Bae, Jae Hyeon <[EMAIL PROTECTED]>
> >> wrote:
> >> > Hi
> >> >
> >> > I am wondering that we do really need the following producer
> >> configuration
> >> >
> >> >   val reconnectInterval = Utils.getInt(props, "reconnect.interval",
> >> 30000)
> >> >
> >> >   /** negative reconnect time interval means disabling this time-based
> >> > reconnect feature */
> >> >   var reconnectTimeInterval = Utils.getInt(props,
> >> > "reconnect.time.interval.ms", 1000*1000*10)
> >> >
> >> > If these configurations have the history, please let me know.
> >> > Otherwise, I want to disable these default configurations.
> >> >
> >> > Thank you
> >> > Best, Jae
> >>
>