Kafka, mail # user - Re: Number of Partitions Per Broker - 2013-01-15, 16:22
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Number of Partitions Per Broker
At LinkedIn, some of  the high volume topics are configured with more than
1 partition per broker. Having more partitions increases I/O parallelism
for writes and also increases the degree of parallelism for consumers
(since partition is the unit for distributing data to consumers). On the
other hand, more partitions adds some overhead: (a) there will be more
files and thus more open file handlers; (b) there are more offsets to be
checkpointed by consumers which can increase the load of ZK. So, you want
to balace these tradeoffs.

Thanks,

Jun

On Mon, Jan 14, 2013 at 11:55 PM, Andrew Psaltis <
[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB