Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Re: Is it possible to get latest offset from kafka server?

Copy link to this message
Re: Is it possible to get latest offset from kafka server?
Of course -- make an Offset Request. This can be done in many ways,
Java, Python, C++, Ruby. -1 means "get latest offset", if I remember


It's just bytes on the wire, and bytes come back.


On Sat, Jun 29, 2013 at 10:56 AM, Yu, Libo <[EMAIL PROTECTED]> wrote:
> Hi,
> I know kafka.tools.ConsumerOffsetChecker can find out the latest offset for
> a topic from log files internally. Is there any public API that users can use for that?
> This would be useful for the failover scenario. When one cluster is down, the
> publisher will publish to the backup cluster. But the backup cluster always lags
> behind. If it is possible for publisher to find out the latest offset from the backup,
> it can always publish from that offset assuming publisher has a large enough
> buffer.  This guarantees no message loss. Thanks.
> Libo

Jun Rao 2013-07-01, 03:52