Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> are topics and partitions dynamic?


+
S Ahmed 2013-01-27, 20:24
+
Neha Narkhede 2013-01-27, 20:37
+
S Ahmed 2013-01-27, 20:40
Copy link to this message
-
Re: are topics and partitions dynamic?
In 0.8, when a topic is created, we decide how each replica of a partition
is mapped to a broker. In general, we try to spread the replicas across all
brokers.

Thanks,

Jun

On Sun, Jan 27, 2013 at 12:40 PM, S Ahmed <[EMAIL PROTECTED]> wrote:

> So from what I understand, a single broker will contain all the messages
> for a given topic (and its partitions).  Who decides which broker will own
> a particular topic?
>
>
> On Sun, Jan 27, 2013 at 3:36 PM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > In Kafka 0.7, topics and partitions are dynamic in the sense that a
> > partition is created when the broker receives the first message for that
> > partition.
> > In Kafka 0.8, there are 2 ways of creating a new topic -
> > 1. Turn on auto.create.topics.enable option on the broker. When the
> broker
> > receives the first message for a new topic, it creates that topic with
> > num.partitions and default.replication.factor.
> > 2. Use the admin command bin/kafka-create-topic.sh
> >
> > Thanks,
> > Neha
> >
> >
> > On Sun, Jan 27, 2013 at 12:23 PM, S Ahmed <[EMAIL PROTECTED]> wrote:
> >
> > > I'm looking at the kafka server.properties that is in /config folder
> and
> > > didn't find any reference to topics or partitions.
> > >
> > > Are topics and optionally partitions dynamic in nature or do they have
> to
> > > be defined before starting the broker? (and consequently producers
> can't
> > > send to a topic and partition that isn't predefined).
> > >
> >
>

 
+
Noel Golding 2013-05-23, 15:04