Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Kafka cluster with lots of topics


+
Joe Freeman 2013-11-13, 12:49
Copy link to this message
-
Re: Kafka cluster with lots of topics
At those many topics, zookeeper will be the main bottleneck. Leader
election process will take very long increasing the unavailability window
of the cluster.

Thanks,
Neha
On Nov 13, 2013 4:49 AM, "Joe Freeman" <[EMAIL PROTECTED]> wrote:

> Would I be correct in assuming that a Kafka cluster won't scale well to
> support lots (tens of millions) of topics? If I understand correctly, a
> node being added or removed would involve a leader election for each topic,
> which is a relatively expensive operation?
>

 
+
hsy541@...> 2013-11-13, 19:50
+
Neha Narkhede 2013-11-13, 22:55
+
Edward Capriolo 2013-11-14, 04:38
+
Joe Freeman 2013-11-14, 14:19
+
Robert Rodgers 2013-11-18, 19:57