Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - async producer behavior if zk and/or kafka cluster goes away...


Copy link to this message
-
Re: async producer behavior if zk and/or kafka cluster goes away...
Jason Rosenberg 2012-11-20, 18:04
On Tue, Nov 20, 2012 at 10:00 AM, Neha Narkhede <[EMAIL PROTECTED]>wrote:

> > By requiring use of a configured broker.list for each client, means that
> > 1000's of deployed apps need to be updated any time the kafka cluster
> > changes, no?  (Or am I not understanding?).
>
> The advantage is that you can configure broker.list to point to a VIP, so
> you
> can transparently change the brokers behind the VIP without having to
> re-configure
> the producers.
Yeah, we've been hoping not to use VIP's for this sort of thing, but I
suppose this makes sense.
> On the other hand, if you ever had to make a similar
> change to your
> zookeeper cluster, it will be very operationally heavy since you will
> have to make a
> config change on each of your producers.
Good point (but zk's will need high sla's anyway, for other reasons).
> > You mention that auto-discovery of new brokers will still work, is that
> > dependent on the existing configured broker.list set still being
> available
> > also?
>
> It depends on at least one broker in the cluster being alive. If none
> of them are alive,
> you have a much bigger problem to worry about.
>

Ok, that's what I figured.  So it's fine unless we decide to move all the
brokers to a new subnet, or something.

Jason