Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Failed to produce message from the second producer


Copy link to this message
-
Re: Failed to produce message from the second producer
Please specify both the brokers in same order for each producer as follows:
kafkaBrokerList=<KAFKA_HOST1>:<PORT>,<KAFKA_HOST2>:<PORT>

The reason I suspect second broker is not functioning as expected because
broker 0 is leader for both partitions as well as it is the only isr.
Assuming producer on second machine can connect to broker on first machine,
it should be able to publish to the first broker which is leader.

You could also check in zookeeper broker node registry to ensure both
brokers are in fact available.

/brokers/ids/[0...N] --> host:port (ephemeral node)

and broker topic registry

/brokers/topics/[topic]/[0...N] --> nPartions (ephemeral node)
On Mon, Nov 11, 2013 at 10:39 PM, ji yan <[EMAIL PROTECTED]> wrote:

> @Priya, I have two brokers, each running on each of two machines. The
> producer broker-list is pointing to the broker server on the same machine
> as the producer.
>
> @Jun, I created the topic using kafka-create-topic.sh, the error is
> consistently reproduceble
>
>
> On Mon, Nov 11, 2013 at 9:51 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Are those ERROR transient? If so, this is normal when the topic didn't
> > exist and was auto created.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Mon, Nov 11, 2013 at 12:12 AM, ji yan <[EMAIL PROTECTED]> wrote:
> >
> > > Hi Kafka Users
> > >
> > > I have a test setup at home with one machine hosting a zookeeper server
> > and
> > > a broker server, while the second machine hosting a broker server.
> > >
> > > On the first machine, producing and consuming messages are ok. But on
> the
> > > second one, producing message throws this error in the log.
> > >
> > > The topic test has two partitions and two replicas.
> > >
> > > [2013-11-11 00:07:24,974] WARN Error while fetching metadata
> > > [{TopicMetadata for topic test ->
> > > No partition metadata for topic test due to
> > > kafka.common.LeaderNotAvailableException}] for topic [test]: class
> > > kafka.common.LeaderNotAvailableException
> > >  (kafka.producer.BrokerPartitionInfo)
> > > [2013-11-11 00:07:24,975] ERROR Failed to collate messages by topic,
> > > partition due to: Failed to fetch topic metadata for topic: test
> > > (kafka.producer.async.DefaultEventHandler)
> > >
> > > However, when displaying list of topics, I do see a leader for the
> topic
> > >
> > > topic: test partition: 0 leader: 0 replicas: 0,1 isr: 0
> > > topic: test partition: 1 leader: 0 replicas: 1,0 isr: 0
> > >
> > > Thanks
> > >
> >
>
>
>
> --
> Do not wait to strike till the iron is hot; but make it hot by striking
>
> LinkedIn Profile: http://www.linkedin.com/pub/ji-yan/a/a57/79
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB