Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Best partition configuration


Copy link to this message
-
Re: Best partition configuration
Philip O'Toole 2013-08-21, 18:26
1 topic.

I don't understand the second question.

Philip

On Aug 21, 2013, at 9:52 AM, Tom Brown <[EMAIL PROTECTED]> wrote:

> Philip,
>
> How many topics per broker (just one?) And what is the read/write profile
> of your setup?
>
> --Tom
>
>
> On Wed, Aug 21, 2013 at 12:24 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
>
>> On Wed, Aug 21, 2013 at 10:06:50AM -0400, Tom Brown wrote:
>>> I have not been able to find reliable advice regarding how many
>> partitions
>>> should exist on a single broker. How many partitions have you used, and
>>> what kind of throughput have you seen?
>>
>> We settled on 8 per broker, and usually run 3 brokers in production
>> environments, giving us a total of 24 partitions. Throughput has been
>> superb.
>>
>> For integration testing however, we usually use just 1 or 2 partitions.
>>
>> Philip
>>
>>>
>>> Thanks in advance!
>>>
>>> --Tom
>>
>> --
>> Philip O'Toole
>>
>> Senior Developer
>> Loggly, Inc.
>> San Francisco, CA.
>> www.loggly.com
>>
>> Come join us!
>> http://loggly.com/company/careers/
>>