Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Hardware profile


+
Ian Friedman 2013-03-29, 15:33
+
Jun Rao 2013-03-29, 16:44
+
Ian Friedman 2013-03-29, 18:38
+
Neha Narkhede 2013-03-29, 18:46
Copy link to this message
-
Re: Hardware profile
David Arthur 2013-03-29, 18:48
How many brokers are you (LinkedIn) running? What kind of network topology?

On 3/29/13 2:45 PM, Neha Narkhede wrote:
> 1. We never share zookeeper and broker on the same hardware. Both need
> significant memory to operate efficiently.
> 2. 14 drive setup is just for Kafka. We have a separate disk for the OS, AFAIK.
>
> Thanks,
> Neha
>
> On Fri, Mar 29, 2013 at 11:37 AM, Ian Friedman <[EMAIL PROTECTED]> wrote:
>> Thanks Jun. Couple more questions:
>> 1. Do you guys have dedicated hardware for Zookeeper or do you have a few machines run both a ZK and a broker? If so, do you keep the ZK and Kafka data on separate volumes?
>> 2. You use the 14 drive raid setup is just for Kafka data and a separate drive for the OS?
>>
>> Thanks again,
>> Ian
>>
>>
>> On Friday, March 29, 2013 at 12:43 PM, Jun Rao wrote:
>>
>>> It's more or less the same. Our new server has 14 sata disks, each of 1 TB.
>>> The disk also has better write latency due to larger write cache.
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>> On Fri, Mar 29, 2013 at 8:32 AM, Ian Friedman <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I'm wondering how up to date the hardware specs listed on this page are:
>>>> https://cwiki.apache.org/confluence/display/KAFKA/Operations
>>>>
>>>> We're evaluating hardware for a Kafka broker/ZK quorum buildout and
>>>> looking for some tips and/or sample configurations if anyone can help us
>>>> out with some recommendations.
>>>>
>>>> Thanks in advance,
>>>> Ian
>>>>
>>>
>>>
>>
 
+
Jun Rao 2013-03-29, 22:54
+
David Arthur 2013-03-29, 23:44
+
Neha Narkhede 2013-03-29, 23:54