I have a question regarding ordering of consumed messages. We timestamp our
messages and send them into Kafka in order. I wrote a simple consumer that
simply consumes the messages and prints out the timestamp. I see messages
for all seven days worth of date being consumed at once.
5 Kafka brokers
The consumer has 10 threads, simply connects, consumes and prints
timestamps. It is set to the "smallest" offset so that it reads from the
beginning. There are many millions of messages so I think I can rule out
some partitions not having messages for certain days as the cause. I know
that Kafka doesn't guarantee ordering across partitions but I would assume
that with this volume of messages I would see the timestamps for the first
day, followed by the second day, etc. Instead I see them all print at once.
Any ideas what I might be doing wrong?