Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Re: heterogenous kafka cluster?


+
Francis Dallaire 2013-05-17, 23:28
+
Jason Rosenberg 2013-05-17, 21:30
+
Neha Narkhede 2013-05-17, 21:35
+
Jason Rosenberg 2013-05-17, 21:48
+
Chris Riccomini 2013-05-17, 21:50
+
Jason Rosenberg 2013-05-17, 22:26
Copy link to this message
-
Re: heterogenous kafka cluster?
Jun Rao 2013-05-18, 03:38
In 0.8, you can create topics manually and explicitly specify the replica
to broker mapping. Post 0.8, we can think of some more automated ways to
deal with this (e.g., let each broker carry some kind of weight).

Thanks,

Jun
On Fri, May 17, 2013 at 2:29 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I'm wondering if there's a good way to have a heterogenous kafka cluster
> (specifically, if we have nodes with different sized disks).  So, we might
> want a larger node to receive more messages than a smaller node, etc.
>
> I expect there's something we can do with using a partitioner that has
> specific knowledge about the hosts in the cluster, but this feels messy, to
> have this config on every producer client....
>
> Thoughts?
>
> Jason
>

 
+
Jason Rosenberg 2013-05-18, 06:28
+
Neha Narkhede 2013-05-23, 21:31
+
Maxime Brugidou 2013-05-23, 21:37