Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Flush configuration per topic


Copy link to this message
-
Re: Flush configuration per topic
Ok cool, thanks Jun :)

--
Felix
On Fri, Oct 25, 2013 at 11:34 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Yes, there won't be any feature development in 0.7.
>
> Thanks,
>
> Jun
>
>
> On Fri, Oct 25, 2013 at 6:32 PM, Felix GV <[EMAIL PROTECTED]> wrote:
>
> > Hi guys,
> >
> > I don't see log.flush.interval being available on per-topic basis in
> 0.7.2
> > either (only the time interval is configurable per topic), as in 0.8.
> >
> > Is it correct to say that this is only going to be available in 0.8.1 and
> > isn't available in 0.7.x?
> >
> > Thanks a lot guys (: ...
> >
> > --
> > Felix
> >
> >
> > On Fri, Oct 18, 2013 at 10:27 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> >
> > > Both.
> > >
> > > On Thursday, October 17, 2013, Steve Morin wrote:
> > >
> > > > Is there only time delay or can it be set to flush for every message
> > with
> > > > the obvious performance hit?
> > > >
> > > >
> > > > On Wed, Oct 16, 2013 at 9:49 AM, Jay Kreps <[EMAIL PROTECTED]
> > > <javascript:;>>
> > > > wrote:
> > > >
> > > > > Yes, the change in trunk is that all log configurations are
> > > automatically
> > > > > available at both the log level and the global default level and
> can
> > be
> > > > set
> > > > > at topic creation time or changed later without bouncing any
> servers.
> > > > >
> > > > > -Jay
> > > > >
> > > > >
> > > > > On Tue, Oct 15, 2013 at 5:47 PM, Simon Hørup Eskildsen
> > > > > <[EMAIL PROTECTED] <javascript:;>>wrote:
> > > > >
> > > > > > Do you mean that it's possible to override log configurations per
> > > topic
> > > > > in
> > > > > > trunk?
> > > > > >
> > > > > > Yeah, you're right. :-) I wasn't sure what to call it if not
> > > > consistency,
> > > > > > even though I know that sort of has another meaning in this
> > context.
> > > > > >
> > > > > >
> > > > > > On Tue, Oct 15, 2013 at 6:53 PM, Jay Kreps <[EMAIL PROTECTED]
> > > <javascript:;>>
> > > > wrote:
> > > > > >
> > > > > > > Yeah, looks like you are right, we don't have the per-topic
> > > override
> > > > in
> > > > > > 0.8
> > > > > > > :-(
> > > > > > >
> > > > > > > All log configurations are overridable in trunk which will be
> > > 0.8.1.
> > > > > > >
> > > > > > > Just to be totally clear this setting does not impact
> consistency
> > > > (i.e.
> > > > > > all
> > > > > > > replicas will have the same messages in the same order), nor
> even
> > > > > > > durability (if you have replication > 1), but just
> recoverability
> > > on
> > > > a
> > > > > > > single server in the event of a hard machine crash.
> > > > > > >
> > > > > > > -Jay
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Oct 15, 2013 at 2:07 PM, Simon Hørup Eskildsen
> > > > > > > <[EMAIL PROTECTED] <javascript:;>>wrote:
> > > > > > >
> > > > > > > > 0.8, we're not on master, but we definitely can be.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Oct 15, 2013 at 5:03 PM, Jay Kreps <
> > [EMAIL PROTECTED]
> > > <javascript:;>
> > > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hey Simon,
> > > > > > > > >
> > > > > > > > > What version of Kafka are you using?
> > > > > > > > >
> > > > > > > > > -Jay
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Oct 15, 2013 at 9:56 AM, Simon Hørup Eskildsen
> > > > > > > > > <[EMAIL PROTECTED] <javascript:;>>wrote:
> > > > > > > > >
> > > > > > > > > > Hi Kafkas!
> > > > > > > > > >
> > > > > > > > > > Reading through the documentation and code of Kafka, it
> > seems
> > > > > there
> > > > > > > is
> > > > > > > > no
> > > > > > > > > > feature to set flushing interval (messages/time) for a
> > > specific
> > > > > > > topic.
> > > > > > > > I
> > > > > > > > > am
> > > > > > > > > > interested in this to get consistency for certain topics
> by
> > > > > > flushing
> > > > > > > > > after
> > > > > > > > > > every message, while having eventual consistency for
> other
> > > > topics
> > > > > > > > > > (default).
> > > > > > > > > >
> > > > > > > > > > Would there be interest in such feature? Then I might be