Kafka, mail # user - Re: Limit on number of Kafka Topic - 2013-03-27, 16:49
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Plain View
+
Suyog Rao 2013-03-27, 06:52
+
Jun Rao 2013-03-27, 15:33
+
Jason Huang 2013-03-27, 15:37
+
Jun Rao 2013-03-27, 16:14
Copy link to this message
-
Re: Limit on number of Kafka Topic
We've seen big performace degradation when we tested 1024 topics, so we've
opted to go for a much smaller topic count (< 100).

On the read side, I think performance is largely driven by the ability of
the operating system to effectively cache access to #partitions*topic
files. Clearly if you divide your available memory by 1024 topics you'll
have less in the file cache per topic than you would with only 3 topics
(for example).
On Wed, Mar 27, 2013 at 11:14 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
[EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB