Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> zookeeper partition data


+
Jay Kreps 2013-01-17, 03:48
Copy link to this message
-
Re: zookeeper partition data
Using separate ZK paths for replicas assignment and configs seems better
since it allows each of them to be updated independently. Yes, we should
make a path of values stored in ZK for future extension. Could you file a
jira?

Thanks,

Jun

On Wed, Jan 16, 2013 at 7:47 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Why is the node<=>partition mapping stored as strings?
>
> get /brokers/topics/my-topic
> { "0": ["0"] }
>
> This is a bug, right? Partitions and node ids are ints.
>
> Also, this structure is not very extensible. If you add anything to this
> map that is not a partition-node mapping it would break. I think the way
> this should have been done is
>
> {
>   "partitions:{0:[1,3,2], 1:[3,1,7], ...}
> }
>
> Then we could add more attributes to the map:
>
> {
>   "partitions:{0:[1,3,2], 1:[3,1,7], ...}
>   "settings":{
>      "log.segment.bytes":1234234,
>       ...
>    }
> }
>
> I think our zookeeper data structures are another "contract" it would be
> good to document and review these before the 0.8 release. If we want to
> change something now it is really easy but once it is out there all changes
> will need to be backwards compatible since they are persistent and can be
> shared by all brokers (so rolling upgrade will be difficult).
>
> -Jay
>