Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Best partition configuration


Copy link to this message
-
Re: Best partition configuration
On Wed, Aug 21, 2013 at 10:06:50AM -0400, Tom Brown wrote:
> I have not been able to find reliable advice regarding how many partitions
> should exist on a single broker. How many partitions have you used, and
> what kind of throughput have you seen?

We settled on 8 per broker, and usually run 3 brokers in production
environments, giving us a total of 24 partitions. Throughput has been superb.

For integration testing however, we usually use just 1 or 2 partitions.

Philip

>
> Thanks in advance!
>
> --Tom

--
Philip O'Toole

Senior Developer
Loggly, Inc.
San Francisco, CA.
www.loggly.com

Come join us!
http://loggly.com/company/careers/