|
|
-
Reading "first available" offset for the partition?
Radek Gruchalski 2012-11-26, 15:17
Hi everyone,
I have a question regarding an issue related to OffsetOutOfRange exception. I have an app that consumes messages from a topic, this app uses node.js and node-kafka behind. Our data retention policy is 24h and the app in question was down for over 2 days. Not a problem, this is a prototype, however… Now, when reconnecting, we are receiving OffsetOutOfRange. Understandable, I'm only trying to figure out how to find the "first available" offset for a partition. Am I assuming correctly this can be done with JMX only? Any avro/thrift/rpc call that could give me that data?
Kind regards,
Radek Gruchalski [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) | [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) |
[EMAIL PROTECTED]
(mailto:[EMAIL PROTECTED]) 00447889948663
Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.
-
Re: Reading "first available" offset for the partition?
Joel Koshy 2012-11-27, 00:02
You can obtain this information by sending the broker an OffsetRequest. There is also a command-line tool (GetOffsetShell) that can help understand how it works. On Mon, Nov 26, 2012 at 7:17 AM, Radek Gruchalski < [EMAIL PROTECTED]> wrote:
> Hi everyone, > > I have a question regarding an issue related to OffsetOutOfRange exception. > I have an app that consumes messages from a topic, this app uses node.js > and node-kafka behind. > Our data retention policy is 24h and the app in question was down for over > 2 days. Not a problem, this is a prototype, however… > Now, when reconnecting, we are receiving OffsetOutOfRange. Understandable, > I'm only trying to figure out how to find the "first available" offset for > a partition. > Am I assuming correctly this can be done with JMX only? Any > avro/thrift/rpc call that could give me that data? > > Kind regards, > Radek Gruchalski > [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) > | [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) | > [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) > 00447889948663 > > Confidentiality: > This communication is intended for the above-named person and may be > confidential and/or legally privileged. > If it has come to you in error you must take no action based on it, nor > must you copy or show it to anyone; please delete/destroy and inform the > sender immediately. > >
-
Re: Reading "first available" offset for the partition?
Radek Gruchalski 2012-11-27, 21:23
Thank you.
Kind regards,
Radek Gruchalski [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) | [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) |
[EMAIL PROTECTED]
(mailto:[EMAIL PROTECTED]) 00447889948663
Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On Tuesday, 27 November 2012 at 01:02, Joel Koshy wrote:
> You can obtain this information by sending the broker an OffsetRequest. > There is also a command-line tool (GetOffsetShell) that can help understand > how it works. > > > On Mon, Nov 26, 2012 at 7:17 AM, Radek Gruchalski < > [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote: > > > Hi everyone, > > > > I have a question regarding an issue related to OffsetOutOfRange exception. > > I have an app that consumes messages from a topic, this app uses node.js > > and node-kafka behind. > > Our data retention policy is 24h and the app in question was down for over > > 2 days. Not a problem, this is a prototype, however… > > Now, when reconnecting, we are receiving OffsetOutOfRange. Understandable, > > I'm only trying to figure out how to find the "first available" offset for > > a partition. > > Am I assuming correctly this can be done with JMX only? Any > > avro/thrift/rpc call that could give me that data? > > > > Kind regards, > > Radek Gruchalski > > [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) > > | [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) | > > [EMAIL PROTECTED] (mailto:[EMAIL PROTECTED]) > > 00447889948663 > > > > Confidentiality: > > This communication is intended for the above-named person and may be > > confidential and/or legally privileged. > > If it has come to you in error you must take no action based on it, nor > > must you copy or show it to anyone; please delete/destroy and inform the > > sender immediately. > > > > >
|
|