Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - [kafka-users] how to make Transaction get with the consumer High-level API


Copy link to this message
-
Re: [kafka-users] how to make Transaction get with the consumer High-level API
Joel Koshy 2012-06-19, 17:17
I don't think we expose any offset information in the high-level API in
0.7.1 right? We have MessageAndMetadata but the only metadata exposed is
the topic the message comes from.

Thanks,

Joel

On Tue, Jun 19, 2012 at 9:22 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Jun, I think what he is saying is he wants to keep the offset in the
> database so he can commit the data and offset transactionally.
>
> Tao, in the 0.7 release the offset is not exposed in the high-level API.
> However in 0.7.1 which is currently in RC we do expose this. My
> recommendation would be to give that a try.
>
> -Jay
>
> On Tue, Jun 19, 2012 at 9:16 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Tao,
> >
> > You can use manual offset commit in the high level consumer. Basically,
> > call offset commit only after the data has been successfully processed.
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, Jun 18, 2012 at 10:41 PM, tao li <[EMAIL PROTECTED]>
> wrote:
> >
> > > hi, all
> > >
> > > In my project, we want to consume the msg in kafka in the transaction,
> > so,
> > > when the transaction rollback, we need the kafka offset back to the
> > > point transaction begin, I use the consumer Low-level API make it, it's
> > too
> > > complex, how to  make Transaction get with the consumer High-level API
> if
> > > is possible.
> > >
> > > Best regards,
> > >
> > > litao
> > >
> >
>