Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> SimpleConsumer throwing OffsetOutOfRangeException


Copy link to this message
-
Re: SimpleConsumer throwing OffsetOutOfRangeException
Thanks Joel.

I actually dont want to deal with offsets. The Kafka storm spout is forcing
me to deal with it :)

Regards,
Vaibhav,
GumGum
On Jul 3, 2012 6:51 PM, "Joel Koshy" <[EMAIL PROTECTED]> wrote:

> > The retention period is set to 72 hours.
> >
> > When I run a SimpleConsumer with offset 0, I get
> OffsetOutOfRangeException.
> >
>
> That offset is probably no longer available.
>
>
> > I tried using some arbitrary higher values such as 1000000 etc, but it
> > didn't work.
> >
>
> The offsets need to be at valid message boundaries.
>
>
> > It was working fine few days ago. If I use Zookeeper based consumer, it
> > works fine.
> >
> > Is there any way I can find out what is the least offset for a topic on
> the
> > broker and start with that using  SimpleConsumer? If I don't know the
> exact
> > offset, am I not allowed to use SimpleConsumer?
> >
> >
> You can use SimpleConsumer if you wish although it is more difficult to use
> (since you have to deal with offsets) than the Zookeeper based consumer.
>
> Anyway, to find available offsets you can use the GetOffsetShell:
>
> kafka-run-class.sh kafka.tools.GetOffsetShell --server kafka://broker:port
> --topic topic --partition 0 --time -2
> -2 means earliest offset available on this broker
> -1 means latest offset available on this broker
>
> Thanks,
>
> Joel
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB