Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Topic messages with partitions=1 stored on multiple brokers

Copy link to this message
Re: Topic messages with partitions=1 stored on multiple brokers
Seems to me you are confusing partitions and brokers. Partition count has
nothing to do with the number of brokers to which a message a sent -- just
the number of partitions into which that message will be split when it gets
to a broker.

You need to explicitly set the destination brokers in the Producer, not via
setting "partition" count.

On Fri, Sep 20, 2013 at 1:53 AM, Suchi Amalapurapu <[EMAIL PROTECTED]>wrote:

> My current setup has 2 brokers, multiple topics, each topic with 1
> partition, kafka 0.7.1
> Some topics are logged onto multiple brokers inspite of partitions set to
> 1. Kafka 0.7.1 does not have replication turned on.
> Why are the messages being logged onto multiple brokers when the number of
> partitions is set to 1?
> I have a consumer with parallelism set to 1 and its missing some of these
> messages.
> Suchi