Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> async producer behavior if zk and/or kafka cluster goes away...


+
Jason Rosenberg 2012-11-19, 21:23
+
Jason Rosenberg 2012-11-19, 21:27
+
Jay Kreps 2012-11-19, 22:31
+
Jason Rosenberg 2012-11-19, 23:30
+
Jay Kreps 2012-11-20, 04:50
+
Jason Rosenberg 2012-11-20, 15:57
+
Neha Narkhede 2012-11-20, 16:41
+
Jun Rao 2012-11-20, 17:03
+
Jason Rosenberg 2012-11-20, 17:44
+
Neha Narkhede 2012-11-20, 18:00
Copy link to this message
-
Re: async producer behavior if zk and/or kafka cluster goes away...
On Tue, Nov 20, 2012 at 10:00 AM, Neha Narkhede <[EMAIL PROTECTED]>wrote:

> > By requiring use of a configured broker.list for each client, means that
> > 1000's of deployed apps need to be updated any time the kafka cluster
> > changes, no?  (Or am I not understanding?).
>
> The advantage is that you can configure broker.list to point to a VIP, so
> you
> can transparently change the brokers behind the VIP without having to
> re-configure
> the producers.
Yeah, we've been hoping not to use VIP's for this sort of thing, but I
suppose this makes sense.
> On the other hand, if you ever had to make a similar
> change to your
> zookeeper cluster, it will be very operationally heavy since you will
> have to make a
> config change on each of your producers.
Good point (but zk's will need high sla's anyway, for other reasons).
> > You mention that auto-discovery of new brokers will still work, is that
> > dependent on the existing configured broker.list set still being
> available
> > also?
>
> It depends on at least one broker in the cluster being alive. If none
> of them are alive,
> you have a much bigger problem to worry about.
>

Ok, that's what I figured.  So it's fine unless we decide to move all the
brokers to a new subnet, or something.

Jason
+
Jay Kreps 2012-11-20, 18:20
+
Jason Rosenberg 2012-11-20, 18:28
+
Neha Narkhede 2012-11-20, 18:35
+
Jun Rao 2012-11-20, 18:39
+
Jason Rosenberg 2012-11-20, 19:09
+
Neha Narkhede 2012-11-20, 19:18