-Re: dumb question about offsets
Neha Narkhede 2012-11-22, 15:33
Yes, in Kafka 0.7, the offset is the byte position of the message in the
log for the topic partition. In Kafka 0.8, each message is assigned a
monotonically increasing, contiguous sequence number per partition,
starting with 1. So each message is addressable using this sequence number
instead of the byte position.
Also, the offset keeps increasing over the lifetime of a cluster, even if
Kafka deletes older log segments.
On Thursday, November 22, 2012, Paul Garner wrote:
> from what I read, the message offset is the byte position of the message in
> the log file that Kafka writes to
> the logs are rotated and eventually deleted by Kafka
> ...does this mean the message offset periodically goes back to start at
> zero again? or the offset keeps increasing for the life of the cluster as
> if it was a single big file back to the beginning of time?