Francis Dallaire 2013-05-17, 23:28
Jason Rosenberg 2013-05-17, 21:30
Neha Narkhede 2013-05-17, 21:35
Jason Rosenberg 2013-05-17, 21:48
-Re: heterogenous kafka cluster?
Chris Riccomini 2013-05-17, 21:50
I have no idea if this would be reasonable, but what about just running
two Kafka processes on the bigger box?
On 5/17/13 2:48 PM, "Jason Rosenberg" <[EMAIL PROTECTED]> wrote:
>Just resource allocation issues. E.g. imagine having an existing kafka
>cluster with one machine spec, and getting access to a few more hosts to
>augment the cluster, which are newer and therefore have twice the disk
>storage. I'd like to seamlessly add them into the cluster, without having
>to replace everything en masse. Thus, it would be nice for the newer ones
>to take proportionally more load based on the relative storage available,
>On Fri, May 17, 2013 at 2:34 PM, Neha Narkhede
>> That does seem a little hacky. But I'm trying to understand the
>> behind having to deploy heterogeneous hardware. What are you trying to
>> achieve or optimize?
>> On Fri, May 17, 2013 at 2:29 PM, Jason Rosenberg <[EMAIL PROTECTED]>
>> > Hi,
>> > I'm wondering if there's a good way to have a heterogenous kafka
>> > (specifically, if we have nodes with different sized disks). So, we
>> > want a larger node to receive more messages than a smaller node, etc.
>> > I expect there's something we can do with using a partitioner that has
>> > specific knowledge about the hosts in the cluster, but this feels
>> > have this config on every producer client....
>> > Thoughts?
>> > Jason
Jason Rosenberg 2013-05-17, 22:26
Jun Rao 2013-05-18, 03:38
Jason Rosenberg 2013-05-18, 06:28
Neha Narkhede 2013-05-23, 21:31
Maxime Brugidou 2013-05-23, 21:37