Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> SimpleConsumer throwing OffsetOutOfRangeException

Copy link to this message
Re: SimpleConsumer throwing OffsetOutOfRangeException
> The retention period is set to 72 hours.
> When I run a SimpleConsumer with offset 0, I get OffsetOutOfRangeException.

That offset is probably no longer available.
> I tried using some arbitrary higher values such as 1000000 etc, but it
> didn't work.

The offsets need to be at valid message boundaries.
> It was working fine few days ago. If I use Zookeeper based consumer, it
> works fine.
> Is there any way I can find out what is the least offset for a topic on the
> broker and start with that using  SimpleConsumer? If I don't know the exact
> offset, am I not allowed to use SimpleConsumer?
You can use SimpleConsumer if you wish although it is more difficult to use
(since you have to deal with offsets) than the Zookeeper based consumer.

Anyway, to find available offsets you can use the GetOffsetShell:

kafka-run-class.sh kafka.tools.GetOffsetShell --server kafka://broker:port
--topic topic --partition 0 --time -2
-2 means earliest offset available on this broker
-1 means latest offset available on this broker