Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Flush configuration per topic


+
Simon Hørup Eskildsen 2013-10-15, 16:56
+
Jay Kreps 2013-10-15, 21:03
+
Simon Hørup Eskildsen 2013-10-15, 21:08
+
Jay Kreps 2013-10-15, 22:54
+
Simon Hørup Eskildsen 2013-10-16, 00:48
Copy link to this message
-
Re: Flush configuration per topic
In 0.8, we do have "log.flush.interval.ms.per.topic" (see
http://kafka.apache.org/documentation.html#brokerconfigs for details).

Thanks,

Jun
On Tue, Oct 15, 2013 at 5:47 PM, Simon Hørup Eskildsen
<[EMAIL PROTECTED]>wrote:

> Do you mean that it's possible to override log configurations per topic in
> trunk?
>
> Yeah, you're right. :-) I wasn't sure what to call it if not consistency,
> even though I know that sort of has another meaning in this context.
>
>
> On Tue, Oct 15, 2013 at 6:53 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:
>
> > Yeah, looks like you are right, we don't have the per-topic override in
> 0.8
> > :-(
> >
> > All log configurations are overridable in trunk which will be 0.8.1.
> >
> > Just to be totally clear this setting does not impact consistency (i.e.
> all
> > replicas will have the same messages in the same order), nor even
> > durability (if you have replication > 1), but just recoverability on a
> > single server in the event of a hard machine crash.
> >
> > -Jay
> >
> >
> > On Tue, Oct 15, 2013 at 2:07 PM, Simon Hørup Eskildsen
> > <[EMAIL PROTECTED]>wrote:
> >
> > > 0.8, we're not on master, but we definitely can be.
> > >
> > >
> > > On Tue, Oct 15, 2013 at 5:03 PM, Jay Kreps <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > Hey Simon,
> > > >
> > > > What version of Kafka are you using?
> > > >
> > > > -Jay
> > > >
> > > >
> > > > On Tue, Oct 15, 2013 at 9:56 AM, Simon Hørup Eskildsen
> > > > <[EMAIL PROTECTED]>wrote:
> > > >
> > > > > Hi Kafkas!
> > > > >
> > > > > Reading through the documentation and code of Kafka, it seems there
> > is
> > > no
> > > > > feature to set flushing interval (messages/time) for a specific
> > topic.
> > > I
> > > > am
> > > > > interested in this to get consistency for certain topics by
> flushing
> > > > after
> > > > > every message, while having eventual consistency for other topics
> > > > > (default).
> > > > >
> > > > > Would there be interest in such feature? Then I might be curious to
> > > give
> > > > > the Log module a dive and see if I can foster enough Scala to add
> > this.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > --
> > > > > Simon
> > > > > http://sirupsen.com/
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Simon
> > > http://sirupsen.com/
> > >
> >
>
>
>
> --
> Simon
> http://sirupsen.com/
>

 
+
Simon Hørup Eskildsen 2013-10-16, 11:17
+
Jay Kreps 2013-10-16, 16:51
+
Steve Morin 2013-10-17, 22:04
+
Jay Kreps 2013-10-18, 14:28
+
Felix GV 2013-10-26, 01:33
+
Jun Rao 2013-10-26, 03:34
+
Felix GV 2013-10-27, 03:59