I recently moved my 0.8.0 cluster to a set of entirely new brokers, and was
surprised to find that the producers did not update their list of brokers
to remove the brokers that were no longer in the cluster. That is, I had
brokers 1,2,3,4,5, added brokers 6,7,8,910, waited a day, and stopped
brokers 1,2,3,4,5. After stopping the original brokers, the producers
continued trying to fetch metadata from them, even though they no longer
appeared the topic metadata. It looks like it was basically the same issue
The documentation for "metadata.broker.list" says "This is for
bootstrapping and the producer will only use it for getting metadata". I,
and apparently others, have interpreted the "this is for bootstrapping"
part to mean that it is only used for the initial metadata fetch, and not
as the sole list of brokers to use to fetch metadata.
Does it make sense to change the documentation to something like "The list
of brokers used to fetch metadata. These brokers must always be available"?
Am I understanding metadata.broker.list correctly? Is it necessary to load
balance these brokers or otherwise make sure it does not refer directly to
any brokers that could be removed?