Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> getting error in fetching topic metadata,when giving message to console producer in Kafka cluster


Copy link to this message
-
getting error in fetching topic metadata,when giving message to console producer in Kafka cluster
Hi,

I had 3 nodes Kafka cluster with default.replication.factor=3 in all the
three nodes.
I create a topic "topic1" with 7 partitions and with rep-factor = 3,my list
topic command gave below o/p:

topic: topic1    partition: 0    leader: 1    replicas: 3,2,1    isr: 1,2,3
topic: topic1    partition: 1    leader: 2    replicas: 1,3,2    isr: 2,1,3
topic: topic1    partition: 2    leader: 3    replicas: 2,1,3    isr: 3,1,2
topic: topic1    partition: 3    leader: 1    replicas: 3,1,2    isr: 1,2,3
topic: topic1    partition: 4    leader: 2    replicas: 1,2,3    isr: 2,1,3
topic: topic1    partition: 5    leader: 3    replicas: 2,3,1    isr: 3,1,2
topic: topic1    partition: 6    leader: 1    replicas: 3,2,1    isr: 1,2,3
Then I removed 2 nodes.

Now I have single node cluster with the same property
default.replication.factor=3.So the list topic command is giving me the
below o/p:

topic: topic1    partition: 0    leader: 1    replicas: 3,2,1    isr: 1
topic: topic1    partition: 1    leader: 1    replicas: 1,3,2    isr: 1
topic: topic1    partition: 2    leader: 1    replicas: 2,1,3    isr: 1
topic: topic1    partition: 3    leader: 1    replicas: 3,1,2    isr: 1
topic: topic1    partition: 4    leader: 1    replicas: 1,2,3    isr: 1
topic: topic1    partition: 5    leader: 1    replicas: 2,3,1    isr: 1
topic: topic1    partition: 6    leader: 1    replicas: 3,2,1    isr: 1

Now I started consoleProducer and give some message ,it is giving me
Warning
WARN Fetching topic metadata with correlation id 8 for topics [Set(topic1)]
from broker [id:0,host:192.168.145.107,port:2181] failed
(kafka.client.ClientUtils$)

pasting here the full stack trace:

[2013-10-17 13:01:25,776] WARN Fetching topic metadata with correlation id
8 for topics [Set(topic2)] from broker
[id:0,host:192.168.145.107,port:2181] failed (kafka.client.ClientUtils$)
java.io.EOFException: Received -1 when reading from channel, socket has
likely been closed.
    at kafka.utils.Utils$.read(Utils.scala:395)
    at
kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
    at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
    at
kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
    at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
    at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:74)
    at
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
    at
kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
    at
kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:79)
    at kafka.utils.Utils$.swallow(Utils.scala:186)
    at kafka.utils.Logging$class.swallowError(Logging.scala:105)
    at kafka.utils.Utils$.swallowError(Utils.scala:45)
    at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:79)
    at
kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
    at
kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
    at
kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
    at scala.collection.immutable.Stream.foreach(Stream.scala:254)
    at
kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
    at
kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
[2013-10-17 13:01:25,778] ERROR fetching topic metadata for topics
[Set(topic2)] from broker
[ArrayBuffer(id:0,host:192.168.145.107,port:2181)] failed
(kafka.utils.Utils$)
kafka.common.KafkaException: fetching topic metadata for topics
[Set(topic2)] from broker
[ArrayBuffer(id:0,host:192.168.145.107,port:2181)] failed
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:67)
    at
kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
    at
kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:79)
    at kafka.utils.Utils$.swallow(Utils.scala:186)
    at kafka.utils.Logging$class.swallowError(Logging.scala:105)
    at kafka.utils.Utils$.swallowError(Utils.scala:45)
    at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:79)
    at
kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
    at
kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
    at
kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
    at scala.collection.immutable.Stream.foreach(Stream.scala:254)
    at
kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
    at
kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
Caused by: java.io.EOFException: Received -1 when reading from channel,
socket has likely been closed.
    at kafka.utils.Utils$.read(Utils.scala:395)
    at
kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
    at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
    at
kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
    at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
    at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:74)
    at
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
    ... 12 more
[2013-10-17 13:01:25,787] ERROR Failed to send requests for topics topic2
with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)
[2013-10-17 13:01:25,794] ERROR Error in han
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB