Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Flush configuration per topic


+
Simon Hørup Eskildsen 2013-10-15, 16:56
+
Jay Kreps 2013-10-15, 21:03
+
Simon Hørup Eskildsen 2013-10-15, 21:08
+
Jay Kreps 2013-10-15, 22:54
+
Simon Hørup Eskildsen 2013-10-16, 00:48
+
Jun Rao 2013-10-16, 04:05
+
Simon Hørup Eskildsen 2013-10-16, 11:17
+
Jay Kreps 2013-10-16, 16:51
+
Steve Morin 2013-10-17, 22:04
+
Jay Kreps 2013-10-18, 14:28
Copy link to this message
-
Re: Flush configuration per topic
Hi guys,

I don't see log.flush.interval being available on per-topic basis in 0.7.2
either (only the time interval is configurable per topic), as in 0.8.

Is it correct to say that this is only going to be available in 0.8.1 and
isn't available in 0.7.x?

Thanks a lot guys (: ...

--
Felix
On Fri, Oct 18, 2013 at 10:27 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Both.
>
> On Thursday, October 17, 2013, Steve Morin wrote:
>
> > Is there only time delay or can it be set to flush for every message with
> > the obvious performance hit?
> >
> >
> > On Wed, Oct 16, 2013 at 9:49 AM, Jay Kreps <[EMAIL PROTECTED]
> <javascript:;>>
> > wrote:
> >
> > > Yes, the change in trunk is that all log configurations are
> automatically
> > > available at both the log level and the global default level and can be
> > set
> > > at topic creation time or changed later without bouncing any servers.
> > >
> > > -Jay
> > >
> > >
> > > On Tue, Oct 15, 2013 at 5:47 PM, Simon Hørup Eskildsen
> > > <[EMAIL PROTECTED] <javascript:;>>wrote:
> > >
> > > > Do you mean that it's possible to override log configurations per
> topic
> > > in
> > > > trunk?
> > > >
> > > > Yeah, you're right. :-) I wasn't sure what to call it if not
> > consistency,
> > > > even though I know that sort of has another meaning in this context.
> > > >
> > > >
> > > > On Tue, Oct 15, 2013 at 6:53 PM, Jay Kreps <[EMAIL PROTECTED]
> <javascript:;>>
> > wrote:
> > > >
> > > > > Yeah, looks like you are right, we don't have the per-topic
> override
> > in
> > > > 0.8
> > > > > :-(
> > > > >
> > > > > All log configurations are overridable in trunk which will be
> 0.8.1.
> > > > >
> > > > > Just to be totally clear this setting does not impact consistency
> > (i.e.
> > > > all
> > > > > replicas will have the same messages in the same order), nor even
> > > > > durability (if you have replication > 1), but just recoverability
> on
> > a
> > > > > single server in the event of a hard machine crash.
> > > > >
> > > > > -Jay
> > > > >
> > > > >
> > > > > On Tue, Oct 15, 2013 at 2:07 PM, Simon Hørup Eskildsen
> > > > > <[EMAIL PROTECTED] <javascript:;>>wrote:
> > > > >
> > > > > > 0.8, we're not on master, but we definitely can be.
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 15, 2013 at 5:03 PM, Jay Kreps <[EMAIL PROTECTED]
> <javascript:;>
> > >
> > > > wrote:
> > > > > >
> > > > > > > Hey Simon,
> > > > > > >
> > > > > > > What version of Kafka are you using?
> > > > > > >
> > > > > > > -Jay
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Oct 15, 2013 at 9:56 AM, Simon Hørup Eskildsen
> > > > > > > <[EMAIL PROTECTED] <javascript:;>>wrote:
> > > > > > >
> > > > > > > > Hi Kafkas!
> > > > > > > >
> > > > > > > > Reading through the documentation and code of Kafka, it seems
> > > there
> > > > > is
> > > > > > no
> > > > > > > > feature to set flushing interval (messages/time) for a
> specific
> > > > > topic.
> > > > > > I
> > > > > > > am
> > > > > > > > interested in this to get consistency for certain topics by
> > > > flushing
> > > > > > > after
> > > > > > > > every message, while having eventual consistency for other
> > topics
> > > > > > > > (default).
> > > > > > > >
> > > > > > > > Would there be interest in such feature? Then I might be
> > curious
> > > to
> > > > > > give
> > > > > > > > the Log module a dive and see if I can foster enough Scala to
> > add
> > > > > this.
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > > --
> > > > > > > > Simon
> > > > > > > > http://sirupsen.com/
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Simon
> > > > > > http://sirupsen.com/
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Simon
> > > > http://sirupsen.com/
> > > >
> > >
> >
>

 
+
Jun Rao 2013-10-26, 03:34
+
Felix GV 2013-10-27, 03:59