Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> zookeeper partition data

Copy link to this message
zookeeper partition data
Why is the node<=>partition mapping stored as strings?

get /brokers/topics/my-topic
{ "0": ["0"] }

This is a bug, right? Partitions and node ids are ints.

Also, this structure is not very extensible. If you add anything to this
map that is not a partition-node mapping it would break. I think the way
this should have been done is

  "partitions:{0:[1,3,2], 1:[3,1,7], ...}

Then we could add more attributes to the map:

  "partitions:{0:[1,3,2], 1:[3,1,7], ...}

I think our zookeeper data structures are another "contract" it would be
good to document and review these before the 0.8 release. If we want to
change something now it is really easy but once it is out there all changes
will need to be backwards compatible since they are persistent and can be
shared by all brokers (so rolling upgrade will be difficult).


Jun Rao 2013-01-17, 15:59