Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Partitions per topic per broker?


Copy link to this message
-
Re: Partitions per topic per broker?
Ian Friedman 2013-07-29, 17:45
Thanks for the clarification guys.

--
Ian Friedman
On Friday, July 26, 2013 at 1:16 AM, Jun Rao wrote:

> I assume this is 0.7. In 0.7, partitions are per broker. So, if you
> configure 100 partitions, each broker will have 100 partitions. In 0.8,
> partitions is at the cluster level and won't change when new brokers are
> added.
>
> Thanks,
>
> Jun
>
>
> On Thu, Jul 25, 2013 at 11:29 AM, Ian Friedman <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
>
> > Hi guys, apologies in advance for the newb question:
> >
> > I am running a 3 broker setup, and I have a topic configured with 100
> > partitions in the broker config. But I've noticed that what seems to happen
> > is that each broker gets 100 partitions and it looks kind of like this in
> > the consumer logs: 1-0, 1-1, 1-2, ... 1-99, 2-0, 2-1... etc. So the topic
> > actually has 300 partitions, 100 per broker? Why? How does this work with
> > the random partitioning scheme? I don't really understand the design behind
> > this and my reading of the documentation hasn't seemed to turn up anything
> > (though it's entirely possible I missed it).
> >
> > Thanks in advance!
> >
> > --
> > Ian Friedman
> >
>
>
>