Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Re: Is it possible to get latest offset from kafka server?


Copy link to this message
-
Re: Is it possible to get latest offset from kafka server?
Philip O'Toole 2013-06-29, 18:23
Of course -- make an Offset Request. This can be done in many ways,
Java, Python, C++, Ruby. -1 means "get latest offset", if I remember
correctly.

http://people.apache.org/~joestein/kafka-0.7.1-incubating-docs/

It's just bytes on the wire, and bytes come back.

Philip

On Sat, Jun 29, 2013 at 10:56 AM, Yu, Libo <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I know kafka.tools.ConsumerOffsetChecker can find out the latest offset for
> a topic from log files internally. Is there any public API that users can use for that?
>
> This would be useful for the failover scenario. When one cluster is down, the
> publisher will publish to the backup cluster. But the backup cluster always lags
> behind. If it is possible for publisher to find out the latest offset from the backup,
> it can always publish from that offset assuming publisher has a large enough
> buffer.  This guarantees no message loss. Thanks.
>
>
> Libo
>

 
+
Jun Rao 2013-07-01, 03:52