Kafka, mail # user - Re: Kafka cluster with lots of topics - 2013-11-14, 04:38
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Kafka cluster with lots of topics
Zookeeper will not be the only problem. The first is that each topic is a
directory on the file system. Each of those is going to have files inside
it. This is going to be fairly overwhelming for the file system. Also I can
not speak for the internals but there may be cases where this many topics
allocates a big array or some other non-optimal behaviour.

Like a RDBMS with this many tables one might ask, why? Isn't there a way to
design the system multi-tennent where so many physical topics are not
needed?
On Wed, Nov 13, 2013 at 9:41 AM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB