Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Commit Offset per topic


Copy link to this message
-
Re: Commit Offset per topic
Roman Garcia 2013-11-08, 16:10
So, what would be that API for offset commit in 0.8 version? Are there any
docs about it?
On Thu, Nov 7, 2013 at 10:56 PM, 小宇 <[EMAIL PROTECTED]> wrote:

> offset commit API could solve your problem, it's for 0.8 version.
>
> ---Sent from Boxer | http://getboxer.com
>
> Thanks Neha! I guess auto-commit it is for now...
>
>
>
>
>
> On Tue, Nov 5, 2013 at 5:08 AM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
>
>
> > Currently, the only way to achieve that is to use the SimpleConsumer API.
>
> > We are considering the feature you mentioned for the 0.9 release -
>
> >
>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPI
>
> >
>
> > Thanks,
>
> > Neha
>
> >
>
> >
>
> > On Mon, Nov 4, 2013 at 9:57 AM, Roman Garcia <[EMAIL PROTECTED]>
>
> > wrote:
>
> >
>
> > > Right Neha, thanks. I was aware of that particular API, but I process
>
> > > topics within separate threads (actors) so I cannot assume every
>
> > > topic/partition is supposed to commit its offset. Instead, I could use
> a
>
> > > per-stream (topic-partition) commit.
>
> > > Is this use case valid? Should there be a ticket to implement this?
>
> > > Regards,
>
> > > Roman
>
> > >
>
> > >
>
> > > On Mon, Nov 4, 2013 at 1:28 PM, Neha Narkhede <[EMAIL PROTECTED]
>
> > > >wrote:
>
> > >
>
> > > > The API you are looking for is ConsumerConnector.commitOffsets().
>
> > > However,
>
> > > > not that it will commit offsets for all partitions that the
> particular
>
> > > > consumer connector owns at that time. And the set of partitions owned
>
> > by
>
> > > a
>
> > > > particular consumer connector can change over time.
>
> > > >
>
> > > > Thanks,
>
> > > > Neha
>
> > > >
>
> > > >
>
> > > > On Mon, Nov 4, 2013 at 7:58 AM, Roman Garcia <[EMAIL PROTECTED]
> >
>
> > > > wrote:
>
> > > >
>
> > > > > Hi, I'm still using Kafka 0.7, and was wondering if there's a way I
>
> > can
>
> > > > > "manually" commit offsets per Topic/Partition.
>
> > > > >
>
> > > > > I was expecting something like KafkaMessageStream#commitOffset()
>
> > > > > I couldn't find an API for this. If there isn't any, is there a
>
> > reason?
>
> > > > >
>
> > > > > Regards,
>
> > > > > Roman
>
> > > > >
>
> > > >
>
> > >
>
> >
>
>