Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Question about resetting offsets and the high-level consumer


Copy link to this message
-
Re: Question about resetting offsets and the high-level consumer
Found it in archives, autooffset.reset is only used when the consumer
registering first time or the last committed offset is out-of-range, so
that means that we have to create direct connection to the zookeeper and
set/delete the /consumer/xyz/offset/topic123/*-* before starting the
consumer to get it reset to the head of the stream.. shouldn't be a big
problem to patch the consumer api with a method to do that right ?

On 10 November 2012 11:40, Michal Haris <[EMAIL PROTECTED]> wrote:

> Really? So what does the autooffset.reset=largest do ?
> On Nov 9, 2012 3:30 PM, "Jun Rao" <[EMAIL PROTECTED]> wrote:
>
>> Bob,
>>
>> The high level consumer doesn't support resetting the starting offset
>> right
>> now. We will look into that post 0.8 as part of the consumer redesign.
>>
>> Thanks,
>>
>> Jun
>>
>>
>> On Fri, Nov 9, 2012 at 5:04 AM, Bob Cotton <[EMAIL PROTECTED]> wrote:
>>
>> > We have a topic whose consumers, on startup, only need to consume from
>> the
>> > head of the topic. They don't need to consume old messages they may have
>> > missed.
>> >
>> > We would like to use the high-level consumer, as the API is a bit
>> simpler.
>> >
>> > Is there a way to reset the current offsets for the high-level consumer?
>> >
>> > We've though about deleting the consumer group nodes in ZK manually, but
>> > that seems brittle as those locations may change from release to
>> release.
>> >
>> > Is there this level of control somewhere I've missed?
>> >
>> > Thanks
>> > - Bob
>> >
>>
>
--
Michal Haris
Software Engineer

www.visualdna.com | t: +44 (0) 207 734 7033