Hi there this is my first experience w/kafka.  We've deployed it in production (soft release) and using it to create a realtime stream of data--we love it so far.

Running in production we are seeing these types of messages every once in a while:  

[2012-06-09 09:02:36,051] ERROR [pool-1-thread-4] (ConsumerIterator.scala 74) - consumed offset: 22013667532 doesn't match fetch offset: 21008498593 for firehose:1-23: fetched offset = 22013667532: consumed offset = 22013667532;
 Consumer may lose data
[2012-06-09 09:22:48,520] ERROR [pool-1-thread-4] (ConsumerIterator.scala 74) - consumed offset: 21013192930 doesn't match fetch offset: 21475567914 for firehose:1-3: fetched offset = 22021419503: consumed offset = 21013192930;
 Consumer may lose data
[2012-06-09 09:42:34,342] ERROR [pool-1-thread-1] (ConsumerIterator.scala 74) - consumed offset: 21017992042 doesn't match fetch offset: 21477363255 for firehose:1-5: fetched offset = 22029075985: consumed offset = 21017992042;
 Consumer may lose data
[2012-06-09 09:46:50,599] ERROR [pool-1-thread-1] (ConsumerIterator.scala 74) - consumed offset: 21017498912 doesn't match fetch offset: 21476883323 for firehose:1-7: fetched offset = 22022716494: consumed offset = 21017498912;
 Consumer may lose data
[2012-06-09 09:50:54,912] ERROR [pool-1-thread-1] (ConsumerIterator.scala 74) - consumed offset: 21016428723 doesn't match fetch offset: 21475750245 for firehose:1-4: fetched offset = 22027573299: consumed offset = 21016428723;
 Consumer may lose data
[2012-06-09 09:58:29,709] ERROR [pool-1-thread-1] (ConsumerIterator.scala 74) - consumed offset: 21017643906 doesn't match fetch offset: 21477006308 for firehose:1-6: fetched offset = 22025778964: consumed offset = 21017643906;
 Consumer may lose data
[2012-06-09 09:59:04,622] ERROR [pool-1-thread-4] (ConsumerIterator.scala 74) - consumed offset: 21017419393 doesn't match fetch offset: 21476749439 for firehose:1-23: fetched offset = 22025584690: consumed offset = 21017419393;

I am a bit unsure what kafka does when the consumed offset doesn't match the fetch offset.  We are using a pool of threads to consume each stream created by ConsumerConnector.createMessageStreams().  Is this kosher?

Corbin,

This indicates a bug in Kafka. Which version of Kafka are you using? Is
there an easy way to reproduce this problem? Your usage of Kafka consumer
seems to be normal.

Thanks,

Jun

On Tue, Jun 12, 2012 at 9:14 AM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:

> Hi there this is my first experience w/kafka.  We've deployed it in
> production (soft release) and using it to create a realtime stream of
> data--we love it so far.
>
> Running in production we are seeing these types of messages every once in
> a while:
>
> [2012-06-09 09:02:36,051] ERROR [pool-1-thread-4] (ConsumerIterator.scala
> 74) - consumed offset: 22013667532 doesn't match fetch offset: 21008498593
> for firehose:1-23: fetched offset = 22013667532: consumed offset > 22013667532;
>  Consumer may lose data
> [2012-06-09 09:22:48,520] ERROR [pool-1-thread-4] (ConsumerIterator.scala
> 74) - consumed offset: 21013192930 doesn't match fetch offset: 21475567914
> for firehose:1-3: fetched offset = 22021419503: consumed offset > 21013192930;
>  Consumer may lose data
> [2012-06-09 09:42:34,342] ERROR [pool-1-thread-1] (ConsumerIterator.scala
> 74) - consumed offset: 21017992042 doesn't match fetch offset: 21477363255
> for firehose:1-5: fetched offset = 22029075985: consumed offset > 21017992042;
>  Consumer may lose data
> [2012-06-09 09:46:50,599] ERROR [pool-1-thread-1] (ConsumerIterator.scala
> 74) - consumed offset: 21017498912 doesn't match fetch offset: 21476883323
> for firehose:1-7: fetched offset = 22022716494: consumed offset > 21017498912;
>  Consumer may lose data
> [2012-06-09 09:50:54,912] ERROR [pool-1-thread-1] (ConsumerIterator.scala
> 74) - consumed offset: 21016428723 doesn't match fetch offset: 21475750245
> for firehose:1-4: fetched offset = 22027573299: consumed offset > 21016428723;
>  Consumer may lose data
> [2012-06-09 09:58:29,709] ERROR [pool-1-thread-1] (ConsumerIterator.scala
> 74) - consumed offset: 21017643906 doesn't match fetch offset: 21477006308
> for firehose:1-6: fetched offset = 22025778964: consumed offset > 21017643906;
>  Consumer may lose data
> [2012-06-09 09:59:04,622] ERROR [pool-1-thread-4] (ConsumerIterator.scala
> 74) - consumed offset: 21017419393 doesn't match fetch offset: 21476749439
> for firehose:1-23: fetched offset = 22025584690: consumed offset > 21017419393;
>
> I am a bit unsure what kafka does when the consumed offset doesn't match
> the fetch offset.  We are using a pool of threads to consume each stream
> created by ConsumerConnector.createMessageStreams().  Is this kosher?
>
>

Jun,
We are using the latest 0.7-incubating version which looks like it has this patch:
https://issues.apache.org/jira/browse/KAFKA-154

Would it make sense to try trunk? :)
I'll have to dig more to reproduce it--I'm not sure under what conditions we are seeing this (during rebalance? or some other event...)

---
Corbin,

This indicates a bug in Kafka. Which version of Kafka are you using? Is
there an easy way to reproduce this problem? Your usage of Kafka consumer
seems to be normal.

Thanks,

Jun

Yes, could you try trunk?

Thanks,

Jun

On Tue, Jun 12, 2012 at 6:41 PM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:

> Jun,
> We are using the latest 0.7-incubating version which looks like it has
> this patch:
> https://issues.apache.org/jira/browse/KAFKA-154
>
> Would it make sense to try trunk? :)
> I'll have to dig more to reproduce it--I'm not sure under what conditions
> we are seeing this (during rebalance? or some other event...)
>
> ---
> Corbin,
>
> This indicates a bug in Kafka. Which version of Kafka are you using? Is
> there an easy way to reproduce this problem? Your usage of Kafka consumer
> seems to be normal.
>
> Thanks,
>
> Jun
>
>
>