Hello Rajasekar,

In 0.8 producers keep a cache of the partition -> leader_broker_id map
which is used to determine to which brokers should the messages be sent.
After new partitions are added, the cache on the producer has not populated
yet hence it will throw this exception. The producer will then try to
refresh its cache by asking the brokers "who are the leaders of these new
partitions that I do not know of before". The brokers at the beginning also
do not know this information, and will only get this information from
controller which will only propagation the leader information after the
leader elections have all been finished.

If you set num.retries to 3 then it is possible that producer gives up too
soon before the leader info ever propagated to producers, hence to
producers also. Could you try to increase producer.num.retries and see if
the producer can eventually succeed in re-trying?

Guozhang
On Tue, Aug 27, 2013 at 8:53 AM, Rajasekar Elango <[EMAIL PROTECTED]>wrote:

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB