Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - SimpleConsumer throwing OffsetOutOfRangeException


Copy link to this message
-
Re: SimpleConsumer throwing OffsetOutOfRangeException
Vaibhav Puranik 2012-07-04, 02:31
Thanks Joel.

I actually dont want to deal with offsets. The Kafka storm spout is forcing
me to deal with it :)

Regards,
Vaibhav,
GumGum
On Jul 3, 2012 6:51 PM, "Joel Koshy" <[EMAIL PROTECTED]> wrote:

> > The retention period is set to 72 hours.
> >
> > When I run a SimpleConsumer with offset 0, I get
> OffsetOutOfRangeException.
> >
>
> That offset is probably no longer available.
>
>
> > I tried using some arbitrary higher values such as 1000000 etc, but it
> > didn't work.
> >
>
> The offsets need to be at valid message boundaries.
>
>
> > It was working fine few days ago. If I use Zookeeper based consumer, it
> > works fine.
> >
> > Is there any way I can find out what is the least offset for a topic on
> the
> > broker and start with that using  SimpleConsumer? If I don't know the
> exact
> > offset, am I not allowed to use SimpleConsumer?
> >
> >
> You can use SimpleConsumer if you wish although it is more difficult to use
> (since you have to deal with offsets) than the Zookeeper based consumer.
>
> Anyway, to find available offsets you can use the GetOffsetShell:
>
> kafka-run-class.sh kafka.tools.GetOffsetShell --server kafka://broker:port
> --topic topic --partition 0 --time -2
> -2 means earliest offset available on this broker
> -1 means latest offset available on this broker
>
> Thanks,
>
> Joel
>