Kafka, mail # dev - dynamic config zookeeper code - 2013-01-24, 05:41
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Plain View
Copy link to this message
dynamic config zookeeper code
Where should the zk code for the dynamic topic config go? I am not sure I
understand the overall lay of the land well enough to know if I am doing
the right thing.

For instance, what the is KafkaZookeeper? What is its domain of
responsibility? Should all zk code go there? Lots of other zk code seems
elsewhere, KafkaZookeeper just seems to be the znode for broker liveness
(perhaps it should be renamed?). Why does some zk stuff go in
KafkaZookeeper and other stuff elsewhere? Actually in practice lots of it
seems to be stuck in ZkUtils as static methods, is there a reason for this?

Maybe this stuff needs a maintainer? Or does it fall under "replication"?

What I am thinking of doing is making a new class TopicConfigWatcher which
has a readAllLogConfigs() method and a watcher on the config change path.
This class would update a map of configs which would be shared with
LogManager to handle "alter topic". "create topic" can just do the read
from zk in response to the controller. Does this sound reasonable?


Joe Stein 2013-01-24, 06:20
Jay Kreps 2013-02-08, 23:03
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB