Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> kafka-49


But if there are no replicas, a produce with an ACK requested would at least
force a flush to disk I assume?  That would be enough for me right now.
And I agree with Jay's comment, the tradeoff of durability vs. throughput
could be chosen by the producer, either with a different message type or
flag in the request.

As for the offset of the message, I'd really like to be able to know what it
is after I produce so that it can be logged and monitored.  Then if needed
something or someone can quickly grab the latest message produced on a topic
for metrics, monitoring, and debugging if something goes awry, otherwise I
need to have the producer also consume the topic to figure it out.. (the
data I'm transporting is a little more important than logs)

So back to the ACK'ing, I'd willing to submit a patch for at least the
optional flag or new message type to force a flush to disk if it would
help..

thanks,
-jeff
On Wed, Jul 20, 2011 at 8:51 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Jeff,
>
> I was thinking of making the ACK mandatory for the producer. The ACK can be
> sent when the message either hits 1 replicas or multiple replicas,
> depending
> on the setting.
>
> Having the ACK include the starting offset of the message seems reasonable.
> It will be a bit complicated for multisend since multiple offsets have to
> be
> returned. What do you need the offset for?
>
> Thanks,
>
> Jun
>
> On Wed, Jul 20, 2011 at 12:47 PM, Jeffrey Damick <[EMAIL PROTECTED]
> >wrote:
>
> > Is there any current thought around KAFKA-49, for acknowledgement of
> > producers?
> > Will this be optional, a new message type(s)?
> > Will the ack be synchronous or asynchronous or depending on request type?
> >
> > It would be fantastic if the ack contained the starting offset of the
> > message published, and not just the ending.
> >
> > This is quickly becoming an issue for us, so we may be able to provide
> some
> > help in this area..
> >
> >
> > thanks
> > -jeff
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB