Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Consumer state


Copy link to this message
-
Re: Consumer state
So this is handled automatically via the java consumer client? Didn't
realize since I am mainly looking into the ruby client.

On 11/11/11 7:09 PM, Jay Kreps wrote:
> The consumer stores the offset for you in zookeeper. This occurs
> periodically in the background (if you enable auto.commit=true), or when
> you call commit() explicitly.
>
> -Jay
>
> On Fri, Nov 11, 2011 at 6:23 PM, Mark<[EMAIL PROTECTED]>  wrote:
>
>> " rather than store metadata for each message (marking it as consumed,
>> say), we just need to store the "high water mark" for each combination of
>> consumer, topic, and partition"
>>
>> Typically where would one store this offset?
>>
>>
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB