-Re: Is it possible to get latest offset from kafka server?
Philip O'Toole 2013-06-29, 18:23
Of course -- make an Offset Request. This can be done in many ways,
Java, Python, C++, Ruby. -1 means "get latest offset", if I remember
It's just bytes on the wire, and bytes come back.
On Sat, Jun 29, 2013 at 10:56 AM, Yu, Libo <[EMAIL PROTECTED]> wrote:
> I know kafka.tools.ConsumerOffsetChecker can find out the latest offset for
> a topic from log files internally. Is there any public API that users can use for that?
> This would be useful for the failover scenario. When one cluster is down, the
> publisher will publish to the backup cluster. But the backup cluster always lags
> behind. If it is possible for publisher to find out the latest offset from the backup,
> it can always publish from that offset assuming publisher has a large enough
> buffer. This guarantees no message loss. Thanks.