Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Consumer state


Copy link to this message
-
Re: Consumer state
Mark 2011-11-12, 16:59
So this is handled automatically via the java consumer client? Didn't
realize since I am mainly looking into the ruby client.

On 11/11/11 7:09 PM, Jay Kreps wrote:
> The consumer stores the offset for you in zookeeper. This occurs
> periodically in the background (if you enable auto.commit=true), or when
> you call commit() explicitly.
>
> -Jay
>
> On Fri, Nov 11, 2011 at 6:23 PM, Mark<[EMAIL PROTECTED]>  wrote:
>
>> " rather than store metadata for each message (marking it as consumed,
>> say), we just need to store the "high water mark" for each combination of
>> consumer, topic, and partition"
>>
>> Typically where would one store this offset?
>>
>>
>>