Kafka, mail # user - Re: kafka.common.OffsetOutOfRangeException - 2013-03-15, 14:05
 Search Hadoop and all its subprojects:
+
Oleg Ruchovets 2013-11-19, 03:41
+
Christopher Alexander 2013-03-13, 20:53
+
Philip OToole 2013-03-13, 21:04
+
Christopher Alexander 2013-03-13, 21:54
+
Philip OToole 2013-03-13, 21:59
+
Jun Rao 2013-03-14, 04:19
+
Christopher Alexander 2013-03-14, 15:03
+
Christopher Alexander 2013-03-14, 15:23
Copy link to this message
-
Re: kafka.common.OffsetOutOfRangeException
I would appreciate it if someone can provide some guidance on how to handle a consumer offset reset. I know this feature is expected to be baked into 0.8.0 (I'm using 0.7.2). Although I in a local development environment, such an exercise would allow me to understand Kafka better and build a trouble-shooting solution set for an eventual production release.

Thanks gurus. :)

Chris

From: "Christopher Alexander" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Thursday, March 14, 2013 11:22:27 AM
Subject: Re: kafka.common.OffsetOutOfRangeException

OK, I re-reviewed the Kafka design doc and looked at the topic file mytopic-0. It definitely isn't 562949953452239 in size (just 293476). Since I am in a local test configuration, how should I resolve the offset drift and where:

1. In ZK by wiping a snapshot.XXX file? This would also affect another app that is also using ZK.
2. In Kafka by wiping the mytopic-0 file?

Thanks,

Chris

From: "Christopher Alexander" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Thursday, March 14, 2013 11:02:57 AM
Subject: Re: kafka.common.OffsetOutOfRangeException

Thanks Jun,

I don't mean to be obtuse, but could you please provide an example? Which file should I determine size for?

Thanks,

Chris

From: "Jun Rao" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Thursday, March 14, 2013 12:18:31 AM
Subject: Re: kafka.common.OffsetOutOfRangeException

Chris,

The last offset can be calculated by adding the file size to the name of
the last Kafka segment file. Then you can see if your offset is in the
range.

Thanks,

Jun

On Wed, Mar 13, 2013 at 2:53 PM, Christopher Alexander <
[EMAIL PROTECTED]> wrote:
 
+
Neha Narkhede 2013-03-15, 15:06
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB