Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Best partition configuration


Copy link to this message
-
Re: Best partition configuration
1 topic.

I don't understand the second question.

Philip

On Aug 21, 2013, at 9:52 AM, Tom Brown <[EMAIL PROTECTED]> wrote:

> Philip,
>
> How many topics per broker (just one?) And what is the read/write profile
> of your setup?
>
> --Tom
>
>
> On Wed, Aug 21, 2013 at 12:24 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
>
>> On Wed, Aug 21, 2013 at 10:06:50AM -0400, Tom Brown wrote:
>>> I have not been able to find reliable advice regarding how many
>> partitions
>>> should exist on a single broker. How many partitions have you used, and
>>> what kind of throughput have you seen?
>>
>> We settled on 8 per broker, and usually run 3 brokers in production
>> environments, giving us a total of 24 partitions. Throughput has been
>> superb.
>>
>> For integration testing however, we usually use just 1 or 2 partitions.
>>
>> Philip
>>
>>>
>>> Thanks in advance!
>>>
>>> --Tom
>>
>> --
>> Philip O'Toole
>>
>> Senior Developer
>> Loggly, Inc.
>> San Francisco, CA.
>> www.loggly.com
>>
>> Come join us!
>> http://loggly.com/company/careers/
>>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB