Why is the node<=>partition mapping stored as strings?

get /brokers/topics/my-topic
{ "0": ["0"] }

This is a bug, right? Partitions and node ids are ints.

Also, this structure is not very extensible. If you add anything to this
map that is not a partition-node mapping it would break. I think the way
this should have been done is

{
  "partitions:{0:[1,3,2], 1:[3,1,7], ...}
}

Then we could add more attributes to the map:

{
  "partitions:{0:[1,3,2], 1:[3,1,7], ...}
  "settings":{
     "log.segment.bytes":1234234,
      ...
   }
}

I think our zookeeper data structures are another "contract" it would be
good to document and review these before the 0.8 release. If we want to
change something now it is really easy but once it is out there all changes
will need to be backwards compatible since they are persistent and can be
shared by all brokers (so rolling upgrade will be difficult).

-Jay

 

Using separate ZK paths for replicas assignment and configs seems better
since it allows each of them to be updated independently. Yes, we should
make a path of values stored in ZK for future extension. Could you file a
jira?

Thanks,

Jun

On Wed, Jan 16, 2013 at 7:47 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Why is the node<=>partition mapping stored as strings?
>
> get /brokers/topics/my-topic
> { "0": ["0"] }
>
> This is a bug, right? Partitions and node ids are ints.
>
> Also, this structure is not very extensible. If you add anything to this
> map that is not a partition-node mapping it would break. I think the way
> this should have been done is
>
> {
>   "partitions:{0:[1,3,2], 1:[3,1,7], ...}
> }
>
> Then we could add more attributes to the map:
>
> {
>   "partitions:{0:[1,3,2], 1:[3,1,7], ...}
>   "settings":{
>      "log.segment.bytes":1234234,
>       ...
>    }
> }
>
> I think our zookeeper data structures are another "contract" it would be
> good to document and review these before the 0.8 release. If we want to
> change something now it is really easy but once it is out there all changes
> will need to be backwards compatible since they are persistent and can be
> shared by all brokers (so rolling upgrade will be difficult).
>
> -Jay
>