Reading through the documentation and code of Kafka, it seems there is no feature to set flushing interval (messages/time) for a specific topic. I am interested in this to get consistency for certain topics by flushing after every message, while having eventual consistency for other topics (default).
Would there be interest in such feature? Then I might be curious to give the Log module a dive and see if I can foster enough Scala to add this.
Yeah, looks like you are right, we don't have the per-topic override in 0.8
All log configurations are overridable in trunk which will be 0.8.1.
Just to be totally clear this setting does not impact consistency (i.e. all replicas will have the same messages in the same order), nor even durability (if you have replication > 1), but just recoverability on a single server in the event of a hard machine crash.
-Jay On Tue, Oct 15, 2013 at 2:07 PM, Simon Hørup Eskildsen <[EMAIL PROTECTED]>wrote:
Do you mean that it's possible to override log configurations per topic in trunk?
Yeah, you're right. :-) I wasn't sure what to call it if not consistency, even though I know that sort of has another meaning in this context. On Tue, Oct 15, 2013 at 6:53 PM, Jay Kreps <[EMAIL PROTECTED]> wrote: Simon http://sirupsen.com/
Yes, the change in trunk is that all log configurations are automatically available at both the log level and the global default level and can be set at topic creation time or changed later without bouncing any servers.
-Jay On Tue, Oct 15, 2013 at 5:47 PM, Simon Hørup Eskildsen <[EMAIL PROTECTED]>wrote: