Kafka, mail # user - Re: Producer only finding partition on 1 of 2 Brokers, even though ZK shows 1 partition exists on both Brokers? - 2013-06-13, 17:32
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Producer only finding partition on 1 of 2 Brokers, even though ZK shows 1 partition exists on both Brokers?
As an update, this continues to affect us.

First I'd like to note ways in which my issues seems different than
KAFKA-278,

* I did not add a new broker or a new topic, this topic has been in use on
two existing brokers for months
* The topic definitely exists on both brokers. The topic/data directory
exists on both, and as noted above both brokers even show it in ZK

That said, I went ahead and did the "work around" from the ticket, which in
my case basically means "restart the brokers" (because the topic/data
directory already exists).

After bouncing both brokers I *did* get data to both brokers for a while.
I'm not yet sure if this only lasts until I have to restart my *producers*
(as I've had to update them a bit lately), but that is my current guess.

When I start a producer now (both brokers up, data looks exactly like ZK in
original post), I get output like this:

2013-06-13_15:46:11.64496 Broker Topic Path => /brokers/topics
2013-06-13_15:46:11.99990 15:46:11.999 [run-main] INFO
kafka.producer.ProducerPool - Creating async producer for broker id = 1 at
10.10.150.16:9092
2013-06-13_15:46:12.00109 15:46:12.001 [run-main] INFO
kafka.producer.ProducerPool - Creating async producer for broker id = 0 at
10.10.71.113:9092
2013-06-13_15:46:16.77956 15:46:16.779 [ProducerSendThread-1375847990]
INFO  kafka.producer.SyncProducer - Connected to 10.10.150.16:9092 for
producing

The last line repeats as the SyncProducer does its periodic reconnect
thing, but note that it is *always* broker 1 at 10.10.150.16:9092, even
though it seems like broker 0 is "seen."

Thanks for your help!

On Tue, Jun 4, 2013 at 6:24 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB