Kafka, mail # user - Large number of partitions - 2013-12-31, 21:44
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Large number of partitions
I'm looking to create a topic with about 1400 partitions to allow a high
degree of parallel processing. We have 5 brokers so that would be 280
partitions per box. Has anyone done something with this number of
partitions before?

Thanks.

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB