Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> understanding OffsetOutOfRangeException's....


Copy link to this message
-
Re: understanding OffsetOutOfRangeException's....
Do you think you can reproduce this easily?

Thanks,

Jun
On Fri, Jan 10, 2014 at 11:33 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

> well, not currently, as we don't have multiple partitions for the
> topics.....but yes, I understand that would help too....
>
> but, we are using this multiple consumers within a process approach in
> general with much success so far......just was curious about this ERROR I
> was seeing :)
>
>
> On Fri, Jan 10, 2014 at 11:06 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Could you increase parallelism on the consumers?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Jan 9, 2014 at 1:22 PM, Jason Rosenberg <[EMAIL PROTECTED]>
> wrote:
> >
> > > The consumption rate is a little better after the refactoring.  The
> main
> > > issue though, was that we had a mismatch between large and small
> topics.
> >  A
> > > large topic can lag, and adversely affect consumption of other topics,
> so
> > > this is an attempt to isolate topic filtering, and better balance the
> > > consumers for the different topics.
> > >
> > > So, it's definitely working on that score.
> > >
> > > The topic that was lagging (and getting OffsetOutOfRangeExceptions) was
> > > doing that before and after the refactor (and after we started also
> > seeing
> > > the ERROR logging).  But consumption of all other topics is working
> > better
> > > now (almost no lag at all).
> > >
> > > I'm also setting the client.id for each consumer in the process, so
> > that I
> > > can see the individual metrics per consumer.
> > >
> > > Jason
> > >
> > >
> > > On Thu, Jan 9, 2014 at 1:00 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > Does the consumption rate in the client (msg/sec) change
> significantly
> > > > after the refactoring?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Wed, Jan 8, 2014 at 10:44 AM, Jason Rosenberg <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > Yes, it's happening continuously, at the moment (although I'm
> > expecting
> > > > the
> > > > > consumer to catch up soon)....
> > > > >
> > > > > It seemed to start happening after I refactored the consumer app to
> > use
> > > > > multiple consumer connectors in the same process (each one has a
> > > separate
> > > > > topic filter, so should be no overlap between them).  All using the
> > > same
> > > > > consumer group.
> > > > >
> > > > > Could it be a thread safety issue in the ZookeeperConsumerConnector
> > > > (seems
> > > > > unlikely).
> > > > >
> > > > > Jason
> > > > >
> > > > >
> > > > > On Wed, Jan 8, 2014 at 1:04 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > > > >
> > > > > > Normally, if the consumer can't keep up, you should just see the
> > > > > > OffsetOutOfRangeException warning. The offset mismatch error
> should
> > > > never
> > > > > > happen. It could be that OffsetOutOfRangeException exposed a bug.
> > Do
> > > > you
> > > > > > think you can reproduce this easily?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > > On Tue, Jan 7, 2014 at 9:29 PM, Jason Rosenberg <
> [EMAIL PROTECTED]>
> > > > > wrote:
> > > > > >
> > > > > > > Jun,
> > > > > > >
> > > > > > > I'm not sure I understand your question, wrt produced data?
> > > > > > >
> > > > > > > But yes, in general, I believe the consumer is not keeping up
> > with
> > > > the
> > > > > > > broker's deleting the data.  So it's trying to fetch the next
> > batch
> > > > of
> > > > > > > data, but it's last offset is no longer there, etc.  So that's
> > the
> > > > > reason
> > > > > > > for the WARN message, in the fetcher thread.
> > > > > > >
> > > > > > > I'm just not sure I understand then why we don't always see the
> > > > > > > ConsumerIterator error also, because won't there always be
> > missing
> > > > data
> > > > > > > detected there?  Why sometimes and not always?  What's the
> > > > difference?
> > > > > > >
> > > > > > > Jason
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Jan 8, 2014 at 12:07 AM, Jun Rao <[EMAIL PROTECTED]>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB