Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Re: Is it possible to get latest offset from kafka server?


Copy link to this message
-
Re: Is it possible to get latest offset from kafka server?
Of course -- make an Offset Request. This can be done in many ways,
Java, Python, C++, Ruby. -1 means "get latest offset", if I remember
correctly.

http://people.apache.org/~joestein/kafka-0.7.1-incubating-docs/

It's just bytes on the wire, and bytes come back.

Philip

On Sat, Jun 29, 2013 at 10:56 AM, Yu, Libo <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I know kafka.tools.ConsumerOffsetChecker can find out the latest offset for
> a topic from log files internally. Is there any public API that users can use for that?
>
> This would be useful for the failover scenario. When one cluster is down, the
> publisher will publish to the backup cluster. But the backup cluster always lags
> behind. If it is possible for publisher to find out the latest offset from the backup,
> it can always publish from that offset assuming publisher has a large enough
> buffer.  This guarantees no message loss. Thanks.
>
>
> Libo
>

 
+
Jun Rao 2013-07-01, 03:52
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB