Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Flush configuration per topic


Copy link to this message
-
Re: Flush configuration per topic
Both.

On Thursday, October 17, 2013, Steve Morin wrote:

> Is there only time delay or can it be set to flush for every message with
> the obvious performance hit?
>
>
> On Wed, Oct 16, 2013 at 9:49 AM, Jay Kreps <[EMAIL PROTECTED]<javascript:;>>
> wrote:
>
> > Yes, the change in trunk is that all log configurations are automatically
> > available at both the log level and the global default level and can be
> set
> > at topic creation time or changed later without bouncing any servers.
> >
> > -Jay
> >
> >
> > On Tue, Oct 15, 2013 at 5:47 PM, Simon Hørup Eskildsen
> > <[EMAIL PROTECTED] <javascript:;>>wrote:
> >
> > > Do you mean that it's possible to override log configurations per topic
> > in
> > > trunk?
> > >
> > > Yeah, you're right. :-) I wasn't sure what to call it if not
> consistency,
> > > even though I know that sort of has another meaning in this context.
> > >
> > >
> > > On Tue, Oct 15, 2013 at 6:53 PM, Jay Kreps <[EMAIL PROTECTED]<javascript:;>>
> wrote:
> > >
> > > > Yeah, looks like you are right, we don't have the per-topic override
> in
> > > 0.8
> > > > :-(
> > > >
> > > > All log configurations are overridable in trunk which will be 0.8.1.
> > > >
> > > > Just to be totally clear this setting does not impact consistency
> (i.e.
> > > all
> > > > replicas will have the same messages in the same order), nor even
> > > > durability (if you have replication > 1), but just recoverability on
> a
> > > > single server in the event of a hard machine crash.
> > > >
> > > > -Jay
> > > >
> > > >
> > > > On Tue, Oct 15, 2013 at 2:07 PM, Simon Hørup Eskildsen
> > > > <[EMAIL PROTECTED] <javascript:;>>wrote:
> > > >
> > > > > 0.8, we're not on master, but we definitely can be.
> > > > >
> > > > >
> > > > > On Tue, Oct 15, 2013 at 5:03 PM, Jay Kreps <[EMAIL PROTECTED]<javascript:;>
> >
> > > wrote:
> > > > >
> > > > > > Hey Simon,
> > > > > >
> > > > > > What version of Kafka are you using?
> > > > > >
> > > > > > -Jay
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 15, 2013 at 9:56 AM, Simon Hørup Eskildsen
> > > > > > <[EMAIL PROTECTED] <javascript:;>>wrote:
> > > > > >
> > > > > > > Hi Kafkas!
> > > > > > >
> > > > > > > Reading through the documentation and code of Kafka, it seems
> > there
> > > > is
> > > > > no
> > > > > > > feature to set flushing interval (messages/time) for a specific
> > > > topic.
> > > > > I
> > > > > > am
> > > > > > > interested in this to get consistency for certain topics by
> > > flushing
> > > > > > after
> > > > > > > every message, while having eventual consistency for other
> topics
> > > > > > > (default).
> > > > > > >
> > > > > > > Would there be interest in such feature? Then I might be
> curious
> > to
> > > > > give
> > > > > > > the Log module a dive and see if I can foster enough Scala to
> add
> > > > this.
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > > --
> > > > > > > Simon
> > > > > > > http://sirupsen.com/
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Simon
> > > > > http://sirupsen.com/
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Simon
> > > http://sirupsen.com/
> > >
> >
>