Kafka, mail # user - Re: About kafka 0.8 producer zookeeper-based load balancing on per-request basis - 2013-01-15, 19:26
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: About kafka 0.8 producer zookeeper-based load balancing on per-request basis
Hey Guys,

Correct me if I'm wrong, but I believe in 0.8, the producer uses a
metadata request to get topic/partition mappings from the broker. The
broker then interacts with ZK (rather than having the producer do it using
zk.connect).

In the event that the master for a topic/partition fails, a new master
broker will be elected for a given topic/partition pair. When the producer
tries to send to the old broker (which is either dead, or a slave now),
the broker will either not respond, or the response will contain an error
code. In either case, I think the producer will do a new metadata request
(to a broker) to get the latest topic/partition to broker mappings. In
this way, it avoids having to use ZooKeeper: it offloads all ZK work to
the broker.

Cheers,
Chris

On 1/14/13 9:35 PM, "Jun Guo -X (jungu - CIIC at Cisco)" <[EMAIL PROTECTED]>
wrote:

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB