Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Commit Offset per topic


Copy link to this message
-
Re: Commit Offset per topic
Neha Narkhede 2013-11-05, 05:09
Currently, the only way to achieve that is to use the SimpleConsumer API.
We are considering the feature you mentioned for the 0.9 release -
https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPI

Thanks,
Neha
On Mon, Nov 4, 2013 at 9:57 AM, Roman Garcia <[EMAIL PROTECTED]> wrote:

> Right Neha, thanks. I was aware of that particular API, but I process
> topics within separate threads (actors) so I cannot assume every
> topic/partition is supposed to commit its offset. Instead, I could use a
> per-stream (topic-partition) commit.
> Is this use case valid? Should there be a ticket to implement this?
> Regards,
> Roman
>
>
> On Mon, Nov 4, 2013 at 1:28 PM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > The API you are looking for is ConsumerConnector.commitOffsets().
> However,
> > not that it will commit offsets for all partitions that the particular
> > consumer connector owns at that time. And the set of partitions owned by
> a
> > particular consumer connector can change over time.
> >
> > Thanks,
> > Neha
> >
> >
> > On Mon, Nov 4, 2013 at 7:58 AM, Roman Garcia <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Hi, I'm still using Kafka 0.7, and was wondering if there's a way I can
> > > "manually" commit offsets per Topic/Partition.
> > >
> > > I was expecting something like KafkaMessageStream#commitOffset()
> > > I couldn't find an API for this. If there isn't any, is there a reason?
> > >
> > > Regards,
> > > Roman
> > >
> >
>