Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> kafka.common.OffsetOutOfRangeException


+
Christopher Alexander 2013-03-13, 20:53
+
Philip OToole 2013-03-13, 21:04
+
Christopher Alexander 2013-03-13, 21:54
+
Jun Rao 2013-03-14, 04:19
+
Christopher Alexander 2013-03-14, 15:03
+
Christopher Alexander 2013-03-14, 15:23
+
Christopher Alexander 2013-03-15, 14:05
+
Neha Narkhede 2013-03-15, 15:06
Copy link to this message
-
Re: kafka.common.OffsetOutOfRangeException
OK, it depends. What that tells you is that offset 0 is available,
since those files are named after the very first offset available in
the file. However, we cannot be sure if 562949953452239 is there. It's
kind of a large number so I doubt it, but you never know. What size is
that file?

If you haven't read the Kafka design doc, you *should*. It is well
written, and will explain what these files mean.

http://kafka.apache.org/design.html

Philip

On Wed, Mar 13, 2013 at 2:53 PM, Christopher Alexander
<[EMAIL PROTECTED]> wrote:
> Thanks for the reply Phillip. I am new to kafka so please bear with me if I say something that's "noobish".
>
> I am running in a localhost configuration for testing. If I checkout kafka logs:
>
>> cd /tmp/kafka-logs
>> ls mytopic-0 # my topic is present
>> cd mytopic-0
>> ls 00000000000000000000.kafka
>> vi 00000000000000000000.kafka
>
> This reveals a hexstring (or another format). No offset visible.
>
> If I checkout zk:
>
>> cd /tmp/zookeeper/version-2
>> ls # I see log and snapshot files
>> vi filename
>
> This reveals binary data. No offset visible.
>
> Are these the locations for finding the current/recent offsets? Thanks.
>
>
> ----- Original Message -----
> From: "Philip O'Toole" <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Wednesday, March 13, 2013 5:04:01 PM
> Subject: Re: kafka.common.OffsetOutOfRangeException
>
> Is offset 562949953452239, partition 0, actually available on the
> Kafka broker? Have you checked?
>
> Philip
>
> On Wed, Mar 13, 2013 at 1:53 PM, Christopher Alexander
> <[EMAIL PROTECTED]> wrote:
>> Hello All,
>>
>> I am using Node-Kafka to connect to Kafka 0.7.2. Things were working fine but we are experiencing repeated exceptions of the following:
>>
>> [2013-03-13 16:45:14,615] ERROR error when processing request FetchRequest(topic:promoterregack, part:0 offset:562949953452239 maxSize:1048576) (kafka.server.KafkaRequestHandlers)
>> kafka.common.OffsetOutOfRangeException: offset 562949953452239 is out of range
>>         at kafka.log.Log$.findRange(Log.scala:46)
>>         at kafka.log.Log.read(Log.scala:264)
>>         at kafka.server.KafkaRequestHandlers.kafka$server$KafkaRequestHandlers$$readMessageSet(KafkaRequestHandlers.scala:112)
>>         at kafka.server.KafkaRequestHandlers.handleFetchRequest(KafkaRequestHandlers.scala:92)
>>         at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$2.apply(KafkaRequestHandlers.scala:39)
>>         at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$2.apply(KafkaRequestHandlers.scala:39)
>>         at kafka.network.Processor.handle(SocketServer.scala:296)
>>         at kafka.network.Processor.read(SocketServer.scala:319)
>>         at kafka.network.Processor.run(SocketServer.scala:214)
>>         at java.lang.Thread.run(Thread.java:662)
>>
>>
>> Is there some configuration or manual maintenance I need to perform on Kafka to remediate the exception? Thank you in advance for your assistance.
>>
>> Kind regards,
>>
>> Chris Alexander
>> Technical Architect and Engineer
>> Gravy, Inc.

 
+
Oleg Ruchovets 2013-11-19, 03:41
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB