Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Topic messages with partitions=1 stored on multiple brokers


Copy link to this message
-
Re: Topic messages with partitions=1 stored on multiple brokers
Seems to me you are confusing partitions and brokers. Partition count has
nothing to do with the number of brokers to which a message a sent -- just
the number of partitions into which that message will be split when it gets
to a broker.

You need to explicitly set the destination brokers in the Producer, not via
setting "partition" count.

Philip
On Fri, Sep 20, 2013 at 1:53 AM, Suchi Amalapurapu <[EMAIL PROTECTED]>wrote:

> My current setup has 2 brokers, multiple topics, each topic with 1
> partition, kafka 0.7.1
> Some topics are logged onto multiple brokers inspite of partitions set to
> 1. Kafka 0.7.1 does not have replication turned on.
> Why are the messages being logged onto multiple brokers when the number of
> partitions is set to 1?
> I have a consumer with parallelism set to 1 and its missing some of these
> messages.
> Suchi
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB