Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> [kafka-users] how to make Transaction get with the consumer High-level API


Copy link to this message
-
Re: [kafka-users] how to make Transaction get with the consumer High-level API
I don't think we expose any offset information in the high-level API in
0.7.1 right? We have MessageAndMetadata but the only metadata exposed is
the topic the message comes from.

Thanks,

Joel

On Tue, Jun 19, 2012 at 9:22 AM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Jun, I think what he is saying is he wants to keep the offset in the
> database so he can commit the data and offset transactionally.
>
> Tao, in the 0.7 release the offset is not exposed in the high-level API.
> However in 0.7.1 which is currently in RC we do expose this. My
> recommendation would be to give that a try.
>
> -Jay
>
> On Tue, Jun 19, 2012 at 9:16 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Tao,
> >
> > You can use manual offset commit in the high level consumer. Basically,
> > call offset commit only after the data has been successfully processed.
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, Jun 18, 2012 at 10:41 PM, tao li <[EMAIL PROTECTED]>
> wrote:
> >
> > > hi, all
> > >
> > > In my project, we want to consume the msg in kafka in the transaction,
> > so,
> > > when the transaction rollback, we need the kafka offset back to the
> > > point transaction begin, I use the consumer Low-level API make it, it's
> > too
> > > complex, how to  make Transaction get with the consumer High-level API
> if
> > > is possible.
> > >
> > > Best regards,
> > >
> > > litao
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB