Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> async producer behavior if zk and/or kafka cluster goes away...


+
Jason Rosenberg 2012-11-19, 21:23
+
Jason Rosenberg 2012-11-19, 21:27
+
Jay Kreps 2012-11-19, 22:31
+
Jason Rosenberg 2012-11-19, 23:30
+
Jay Kreps 2012-11-20, 04:50
+
Jason Rosenberg 2012-11-20, 15:57
+
Neha Narkhede 2012-11-20, 16:41
+
Jun Rao 2012-11-20, 17:03
+
Jason Rosenberg 2012-11-20, 17:44
+
Neha Narkhede 2012-11-20, 18:00
Copy link to this message
-
Re: async producer behavior if zk and/or kafka cluster goes away...
On Tue, Nov 20, 2012 at 10:00 AM, Neha Narkhede <[EMAIL PROTECTED]>wrote:

> > By requiring use of a configured broker.list for each client, means that
> > 1000's of deployed apps need to be updated any time the kafka cluster
> > changes, no?  (Or am I not understanding?).
>
> The advantage is that you can configure broker.list to point to a VIP, so
> you
> can transparently change the brokers behind the VIP without having to
> re-configure
> the producers.
Yeah, we've been hoping not to use VIP's for this sort of thing, but I
suppose this makes sense.
> On the other hand, if you ever had to make a similar
> change to your
> zookeeper cluster, it will be very operationally heavy since you will
> have to make a
> config change on each of your producers.
Good point (but zk's will need high sla's anyway, for other reasons).
> > You mention that auto-discovery of new brokers will still work, is that
> > dependent on the existing configured broker.list set still being
> available
> > also?
>
> It depends on at least one broker in the cluster being alive. If none
> of them are alive,
> you have a much bigger problem to worry about.
>

Ok, that's what I figured.  So it's fine unless we decide to move all the
brokers to a new subnet, or something.

Jason
+
Jay Kreps 2012-11-20, 18:20
+
Jason Rosenberg 2012-11-20, 18:28
+
Neha Narkhede 2012-11-20, 18:35
+
Jun Rao 2012-11-20, 18:39
+
Jason Rosenberg 2012-11-20, 19:09
+
Neha Narkhede 2012-11-20, 19:18
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB