Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> no partition leader yet for partition 0 (0.8.0)


Copy link to this message
-
no partition leader yet for partition 0 (0.8.0)
Hi,

I noticed several errors when writing to a topic with 5 partitions. It
looks like the data was written to all 3 brokers, but I get the following
errors:

9961 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Metadata for topic
partition [test1, 0] is errornous:
[PartitionMetadata(0,None,WrappedArray(),WrappedArray(),5)]
kafka.common.LeaderNotAvailableException
at sun.reflect.GeneratedConstructorAccessor1.newInstance(Unknown Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
<snip>

9962 [main] DEBUG kafka.producer.async.DefaultEventHandler  - Getting the
number of broker partitions registered for topic: test1
9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Getting broker
partition info for topic test1
9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
partition 0 does not have a leader yet
9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
partition 1 has leader 7
9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
partition 2 has leader 8
9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
partition 3 has leader 9
9963 [main] DEBUG kafka.producer.BrokerPartitionInfo  - Topic test1
partition 4 has leader 1
9963 [main] DEBUG kafka.producer.async.DefaultEventHandler  - Broker
partitions registered for topic: test1 are 0,1,2,3,4

This happens a lot as I write data to the Broker.

Topic was created with:

-bash-3.2$ ./kafka-create-topic.sh --topic test1 --partition 5 --replica 3
--zookeeper localhost:2181

Doing a list of topics shows an empty list for that partition:

[2012-11-27 16:03:35,604] INFO Session establishment complete on server
localhost/127.0.0.1:2181, sessionid = 0x23b4218eccd000b, negotiated timeout
= 30000 (org.apache.zookeeper.ClientCnxn)
[2012-11-27 16:03:35,607] INFO zookeeper state changed (SyncConnected)
(org.I0Itec.zkclient.ZkClient)
topic: test1
PartitionMetadata(0,None,List(),List(),5)
PartitionMetadata(1,Some(id:7,creatorId:10.121.31.57-1354023708335,host:10.121.31.57,port:9092),List(id:7,creatorId:10.121.31.57-1354023708335,host:10.121.31.57,port:9092,
id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093,
id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094),ArrayBuffer(id:7,creatorId:10.121.31.57-1354023708335,host:10.121.31.57,port:9092,
id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093,
id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094),0)
PartitionMetadata(2,Some(id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093),List(id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093,
id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094,
id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092),ArrayBuffer(id:8,creatorId:10.121.31.57-1354023708340,host:10.121.31.57,port:9093,
id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094,
id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092),0)
PartitionMetadata(3,Some(id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094),List(id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094,
id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092,
id:2,creatorId:10.121.31.55-1354023701344,host:10.121.31.55,port:9093),ArrayBuffer(id:9,creatorId:10.121.31.57-1354023944130,host:10.121.31.57,port:9094,
id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092,
id:2,creatorId:10.121.31.55-1354023701344,host:10.121.31.55,port:9093),0)
PartitionMetadata(4,Some(id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092),List(id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092,
id:2,creatorId:10.121.31.55-1354023701344,host:10.121.31.55,port:9093,
id:3,creatorId:10.121.31.55-1354023701345,host:10.121.31.55,port:9094),ArrayBuffer(id:1,creatorId:10.121.31.55-1354023701351,host:10.121.31.55,port:9092,
id:2,creatorId:10.121.31.55-1354023701344,host:10.121.31.55,port:9093,
id:3,creatorId:10.121.31.55-1354023701345,host:10.121.31.55,port:9094),0)
[2012-11-27 16:03:36,005] INFO Terminate ZkClient event thread.
(org.I0Itec.zkclient.ZkEventThread)

My partitioner logic is doing a simple modulo on the # of partitions passed:

    return (int) (organizationId % a_numPartitions);

Did I miss a step setting up the topics?

Thanks,

Chris
+
Jun Rao 2012-11-28, 05:13
+
Chris Curtin 2013-03-04, 19:21
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB