-Re: dumb question about offsets
Neha Narkhede 2012-11-22, 18:30
On Thu, Nov 22, 2012 at 7:37 AM, Paul Garner
<[EMAIL PROTECTED]> wrote:
> sorry, just to be clear... in both 0.7 and 0.8 the offset keeps increasing,
> just in 0.8 it's no longer a byte position?
> On Thu, Nov 22, 2012 at 3:33 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
>> Yes, in Kafka 0.7, the offset is the byte position of the message in the
>> log for the topic partition. In Kafka 0.8, each message is assigned a
>> monotonically increasing, contiguous sequence number per partition,
>> starting with 1. So each message is addressable using this sequence number
>> instead of the byte position.
>> Also, the offset keeps increasing over the lifetime of a cluster, even if
>> Kafka deletes older log segments.
>> On Thursday, November 22, 2012, Paul Garner wrote:
>> > from what I read, the message offset is the byte position of the message
>> > the log file that Kafka writes to
>> > the logs are rotated and eventually deleted by Kafka
>> > ...does this mean the message offset periodically goes back to start at
>> > zero again? or the offset keeps increasing for the life of the cluster as
>> > if it was a single big file back to the beginning of time?