Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Commit Offset per topic


Copy link to this message
-
Re: Commit Offset per topic
So, what would be that API for offset commit in 0.8 version? Are there any
docs about it?
On Thu, Nov 7, 2013 at 10:56 PM, 小宇 <[EMAIL PROTECTED]> wrote:

> offset commit API could solve your problem, it's for 0.8 version.
>
> ---Sent from Boxer | http://getboxer.com
>
> Thanks Neha! I guess auto-commit it is for now...
>
>
>
>
>
> On Tue, Nov 5, 2013 at 5:08 AM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
>
>
> > Currently, the only way to achieve that is to use the SimpleConsumer API.
>
> > We are considering the feature you mentioned for the 0.9 release -
>
> >
>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPI
>
> >
>
> > Thanks,
>
> > Neha
>
> >
>
> >
>
> > On Mon, Nov 4, 2013 at 9:57 AM, Roman Garcia <[EMAIL PROTECTED]>
>
> > wrote:
>
> >
>
> > > Right Neha, thanks. I was aware of that particular API, but I process
>
> > > topics within separate threads (actors) so I cannot assume every
>
> > > topic/partition is supposed to commit its offset. Instead, I could use
> a
>
> > > per-stream (topic-partition) commit.
>
> > > Is this use case valid? Should there be a ticket to implement this?
>
> > > Regards,
>
> > > Roman
>
> > >
>
> > >
>
> > > On Mon, Nov 4, 2013 at 1:28 PM, Neha Narkhede <[EMAIL PROTECTED]
>
> > > >wrote:
>
> > >
>
> > > > The API you are looking for is ConsumerConnector.commitOffsets().
>
> > > However,
>
> > > > not that it will commit offsets for all partitions that the
> particular
>
> > > > consumer connector owns at that time. And the set of partitions owned
>
> > by
>
> > > a
>
> > > > particular consumer connector can change over time.
>
> > > >
>
> > > > Thanks,
>
> > > > Neha
>
> > > >
>
> > > >
>
> > > > On Mon, Nov 4, 2013 at 7:58 AM, Roman Garcia <[EMAIL PROTECTED]
> >
>
> > > > wrote:
>
> > > >
>
> > > > > Hi, I'm still using Kafka 0.7, and was wondering if there's a way I
>
> > can
>
> > > > > "manually" commit offsets per Topic/Partition.
>
> > > > >
>
> > > > > I was expecting something like KafkaMessageStream#commitOffset()
>
> > > > > I couldn't find an API for this. If there isn't any, is there a
>
> > reason?
>
> > > > >
>
> > > > > Regards,
>
> > > > > Roman
>
> > > > >
>
> > > >
>
> > >
>
> >
>
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB