Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> dumb question about offsets

Copy link to this message
Re: dumb question about offsets
Yes, in Kafka 0.7, the offset is the byte position of the message in the
log for the topic partition. In Kafka 0.8, each message is assigned a
monotonically increasing, contiguous sequence number per partition,
starting with 1. So each message is addressable using this sequence number
instead of the byte position.

Also, the offset keeps increasing over the lifetime of a cluster, even if
Kafka deletes older log segments.


On Thursday, November 22, 2012, Paul Garner wrote:

> from what I read, the message offset is the byte position of the message in
> the log file that Kafka writes to
> the logs are rotated and eventually deleted by Kafka
> ...does this mean the message offset periodically goes back to start at
> zero again? or the offset keeps increasing for the life of the cluster as
> if it was a single big file back to the beginning of time?