Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # dev - [jira] [Commented] (KAFKA-554) Move all per-topic configuration into ZK and add to the CreateTopicCommand


+
Jay Kreps 2013-03-05, 01:04
+
Neha Narkhede 2013-03-06, 20:48
+
Neha Narkhede 2013-03-08, 18:42
Copy link to this message
-
[jira] [Commented] (KAFKA-554) Move all per-topic configuration into ZK and add to the CreateTopicCommand
"Jun Rao 2013-03-12, 17:38

    [ https://issues.apache.org/jira/browse/KAFKA-554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600218#comment-13600218 ]

Jun Rao commented on KAFKA-554:
-------------------------------

For debugging purpose, is it useful to be able to get the runtime config values used by each broker (e.g., through jmx)?
                
> Move all per-topic configuration into ZK and add to the CreateTopicCommand
> --------------------------------------------------------------------------
>
>                 Key: KAFKA-554
>                 URL: https://issues.apache.org/jira/browse/KAFKA-554
>             Project: Kafka
>          Issue Type: New Feature
>            Reporter: Jay Kreps
>            Assignee: Jay Kreps
>              Labels: project
>             Fix For: 0.8.1
>
>         Attachments: KAFKA-554-v1.patch, KAFKA-554-v2.patch, KAFKA-554-v3.patch, KAFKA-554-v4.patch
>
>
> We have a number of per-topic configurations that control message retention and flush interval. Here is the list of properties I find in KafkaConfig that appear to be per-topic:
>   topic.log.file.size
>   topic.log.roll.hours
>   topic.log.retention.hours
>   topic.log.retention.size
>   topic.flush.intervals.ms
> Currently we specify these in server.properties. This is not a good solution as it requires a rolling bounce of the cluster to make a change, which just doesn't scale to having hundreds of topics. Also the map encoded in a CSV string is kind of hacky.
> We should move these into ZK in some kind of JSON blob that allows easily adding new per-topic configs and we should remove these from server.properties.
> It would be good to start with a wiki design and get consensus on that first.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira