Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> consumer offset not saved in zk


Copy link to this message
-
Re: consumer offset not saved in zk
You are looking at the wrong path in ZK. The correct path for consumer
offset is /consumers/[groupId]/offsets/[topic]/[partitionId] -> long
(offset). For more details on our ZK layout, see
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper

Thanks,

Jun
On Wed, May 22, 2013 at 11:21 PM, rk vishu <[EMAIL PROTECTED]> wrote:

> Hello All,
>
> I recently started experimenting Kafka for my usecase. I am running 0.8 in
> two node kafka setup.
>
> I produced 20messages using a java program(1 partition with 2 replicas) and
> I am running the consumer code as given in the example
> https://cwiki.apache.org/KAFKA/consumer-group-example.html. Consumer
> consumes all the messages and after 10sec program shuts down. Based on the
> configuration in the example, i am expecting that consumer offsets will be
> saved on ZK. If i start the program again, i should not be consuming the
> same messages again. But i am seeing different behavior. Messages are
> getting replayed. I am not seeing any update in ZK also.
>
> [zk: localhost:2181(CONNECTED) 13] get /consumers/1
> null
> cZxid = 0x8009ff0ee
> ctime = Wed May 22 16:59:21 PDT 2013
> mZxid = 0x8009ff0ee
> mtime = Wed May 22 16:59:21 PDT 2013
> pZxid = 0x8009ff0f4
> cversion = 2
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 0
> numChildren = 2
>
>
> Could any one of you explain me what could be the issue?
>
> Ravi
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB