Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> getting error in fetching topic metadata,when giving message to console producer in Kafka cluster


+
Monika Garg 2013-10-17, 08:51
Copy link to this message
-
Re: getting error in fetching topic metadata,when giving message to console producer in Kafka cluster
I think you provided the wrong port in broker-list. 2181 is the ZK port.
Kafka port defaults to 9092.

Thanks,

Jun
On Thu, Oct 17, 2013 at 1:51 AM, Monika Garg <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I had 3 nodes Kafka cluster with default.replication.factor=3 in all the
> three nodes.
> I create a topic "topic1" with 7 partitions and with rep-factor = 3,my list
> topic command gave below o/p:
>
> topic: topic1    partition: 0    leader: 1    replicas: 3,2,1    isr: 1,2,3
> topic: topic1    partition: 1    leader: 2    replicas: 1,3,2    isr: 2,1,3
> topic: topic1    partition: 2    leader: 3    replicas: 2,1,3    isr: 3,1,2
> topic: topic1    partition: 3    leader: 1    replicas: 3,1,2    isr: 1,2,3
> topic: topic1    partition: 4    leader: 2    replicas: 1,2,3    isr: 2,1,3
> topic: topic1    partition: 5    leader: 3    replicas: 2,3,1    isr: 3,1,2
> topic: topic1    partition: 6    leader: 1    replicas: 3,2,1    isr: 1,2,3
>
>
> Then I removed 2 nodes.
>
> Now I have single node cluster with the same property
> default.replication.factor=3.So the list topic command is giving me the
> below o/p:
>
> topic: topic1    partition: 0    leader: 1    replicas: 3,2,1    isr: 1
> topic: topic1    partition: 1    leader: 1    replicas: 1,3,2    isr: 1
> topic: topic1    partition: 2    leader: 1    replicas: 2,1,3    isr: 1
> topic: topic1    partition: 3    leader: 1    replicas: 3,1,2    isr: 1
> topic: topic1    partition: 4    leader: 1    replicas: 1,2,3    isr: 1
> topic: topic1    partition: 5    leader: 1    replicas: 2,3,1    isr: 1
> topic: topic1    partition: 6    leader: 1    replicas: 3,2,1    isr: 1
>
> Now I started consoleProducer and give some message ,it is giving me
> Warning
> WARN Fetching topic metadata with correlation id 8 for topics [Set(topic1)]
> from broker [id:0,host:192.168.145.107,port:2181] failed
> (kafka.client.ClientUtils$)
>
> pasting here the full stack trace:
>
> [2013-10-17 13:01:25,776] WARN Fetching topic metadata with correlation id
> 8 for topics [Set(topic2)] from broker
> [id:0,host:192.168.145.107,port:2181] failed (kafka.client.ClientUtils$)
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
>     at kafka.utils.Utils$.read(Utils.scala:395)
>     at
>
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
>     at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
>     at
>
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
>     at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
>     at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:74)
>     at
>
> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
>     at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
>     at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
>     at
> kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
>     at
>
> kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:79)
>     at kafka.utils.Utils$.swallow(Utils.scala:186)
>     at kafka.utils.Logging$class.swallowError(Logging.scala:105)
>     at kafka.utils.Utils$.swallowError(Utils.scala:45)
>     at
>
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:79)
>     at
>
> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
>     at
>
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
>     at
>
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
>     at scala.collection.immutable.Stream.foreach(Stream.scala:254)
>     at
>
> kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
>     at
> kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
> [2013-10-17 13:01:25,778] ERROR fetching topic metadata for topics

 
+
Monika Garg 2013-10-18, 05:46
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB