Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Flush configuration per topic


+
Simon Hørup Eskildsen 2013-10-15, 16:56
+
Jay Kreps 2013-10-15, 21:03
+
Simon Hørup Eskildsen 2013-10-15, 21:08
+
Jay Kreps 2013-10-15, 22:54
+
Simon Hørup Eskildsen 2013-10-16, 00:48
+
Jun Rao 2013-10-16, 04:05
+
Simon Hørup Eskildsen 2013-10-16, 11:17
+
Jay Kreps 2013-10-16, 16:51
Copy link to this message
-
Re: Flush configuration per topic
Is there only time delay or can it be set to flush for every message with
the obvious performance hit?
On Wed, Oct 16, 2013 at 9:49 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Yes, the change in trunk is that all log configurations are automatically
> available at both the log level and the global default level and can be set
> at topic creation time or changed later without bouncing any servers.
>
> -Jay
>
>
> On Tue, Oct 15, 2013 at 5:47 PM, Simon Hørup Eskildsen
> <[EMAIL PROTECTED]>wrote:
>
> > Do you mean that it's possible to override log configurations per topic
> in
> > trunk?
> >
> > Yeah, you're right. :-) I wasn't sure what to call it if not consistency,
> > even though I know that sort of has another meaning in this context.
> >
> >
> > On Tue, Oct 15, 2013 at 6:53 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> >
> > > Yeah, looks like you are right, we don't have the per-topic override in
> > 0.8
> > > :-(
> > >
> > > All log configurations are overridable in trunk which will be 0.8.1.
> > >
> > > Just to be totally clear this setting does not impact consistency (i.e.
> > all
> > > replicas will have the same messages in the same order), nor even
> > > durability (if you have replication > 1), but just recoverability on a
> > > single server in the event of a hard machine crash.
> > >
> > > -Jay
> > >
> > >
> > > On Tue, Oct 15, 2013 at 2:07 PM, Simon Hørup Eskildsen
> > > <[EMAIL PROTECTED]>wrote:
> > >
> > > > 0.8, we're not on master, but we definitely can be.
> > > >
> > > >
> > > > On Tue, Oct 15, 2013 at 5:03 PM, Jay Kreps <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > > > Hey Simon,
> > > > >
> > > > > What version of Kafka are you using?
> > > > >
> > > > > -Jay
> > > > >
> > > > >
> > > > > On Tue, Oct 15, 2013 at 9:56 AM, Simon Hørup Eskildsen
> > > > > <[EMAIL PROTECTED]>wrote:
> > > > >
> > > > > > Hi Kafkas!
> > > > > >
> > > > > > Reading through the documentation and code of Kafka, it seems
> there
> > > is
> > > > no
> > > > > > feature to set flushing interval (messages/time) for a specific
> > > topic.
> > > > I
> > > > > am
> > > > > > interested in this to get consistency for certain topics by
> > flushing
> > > > > after
> > > > > > every message, while having eventual consistency for other topics
> > > > > > (default).
> > > > > >
> > > > > > Would there be interest in such feature? Then I might be curious
> to
> > > > give
> > > > > > the Log module a dive and see if I can foster enough Scala to add
> > > this.
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > --
> > > > > > Simon
> > > > > > http://sirupsen.com/
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Simon
> > > > http://sirupsen.com/
> > > >
> > >
> >
> >
> >
> > --
> > Simon
> > http://sirupsen.com/
> >
>

 
+
Jay Kreps 2013-10-18, 14:28
+
Felix GV 2013-10-26, 01:33
+
Jun Rao 2013-10-26, 03:34
+
Felix GV 2013-10-27, 03:59
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB