Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> zookeeper partition data


Copy link to this message
-
Re: zookeeper partition data
Using separate ZK paths for replicas assignment and configs seems better
since it allows each of them to be updated independently. Yes, we should
make a path of values stored in ZK for future extension. Could you file a
jira?

Thanks,

Jun

On Wed, Jan 16, 2013 at 7:47 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Why is the node<=>partition mapping stored as strings?
>
> get /brokers/topics/my-topic
> { "0": ["0"] }
>
> This is a bug, right? Partitions and node ids are ints.
>
> Also, this structure is not very extensible. If you add anything to this
> map that is not a partition-node mapping it would break. I think the way
> this should have been done is
>
> {
>   "partitions:{0:[1,3,2], 1:[3,1,7], ...}
> }
>
> Then we could add more attributes to the map:
>
> {
>   "partitions:{0:[1,3,2], 1:[3,1,7], ...}
>   "settings":{
>      "log.segment.bytes":1234234,
>       ...
>    }
> }
>
> I think our zookeeper data structures are another "contract" it would be
> good to document and review these before the 0.8 release. If we want to
> change something now it is really easy but once it is out there all changes
> will need to be backwards compatible since they are persistent and can be
> shared by all brokers (so rolling upgrade will be difficult).
>
> -Jay
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB