Home | About | Sematext search-lucene.com search-hadoop.com search-devops.com metrics + logs = try SPM and Logsene for free
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Producer only finding partition on 1 of 2 Brokers, even though ZK shows 1 partition exists on both Brokers?


Copy link to this message
-
Re: Producer only finding partition on 1 of 2 Brokers, even though ZK shows 1 partition exists on both Brokers?
As an update, this continues to affect us.

First I'd like to note ways in which my issues seems different than
KAFKA-278,

* I did not add a new broker or a new topic, this topic has been in use on
two existing brokers for months
* The topic definitely exists on both brokers. The topic/data directory
exists on both, and as noted above both brokers even show it in ZK

That said, I went ahead and did the "work around" from the ticket, which in
my case basically means "restart the brokers" (because the topic/data
directory already exists).

After bouncing both brokers I *did* get data to both brokers for a while.
I'm not yet sure if this only lasts until I have to restart my *producers*
(as I've had to update them a bit lately), but that is my current guess.

When I start a producer now (both brokers up, data looks exactly like ZK in
original post), I get output like this:

2013-06-13_15:46:11.64496 Broker Topic Path => /brokers/topics
2013-06-13_15:46:11.99990 15:46:11.999 [run-main] INFO
kafka.producer.ProducerPool - Creating async producer for broker id = 1 at
10.10.150.16:9092
2013-06-13_15:46:12.00109 15:46:12.001 [run-main] INFO
kafka.producer.ProducerPool - Creating async producer for broker id = 0 at
10.10.71.113:9092
2013-06-13_15:46:16.77956 15:46:16.779 [ProducerSendThread-1375847990]
INFO  kafka.producer.SyncProducer - Connected to 10.10.150.16:9092 for
producing

The last line repeats as the SyncProducer does its periodic reconnect
thing, but note that it is *always* broker 1 at 10.10.150.16:9092, even
though it seems like broker 0 is "seen."

Thanks for your help!

On Tue, Jun 4, 2013 at 6:24 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB