Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> understanding OffsetOutOfRangeException's....


Copy link to this message
-
Re: understanding OffsetOutOfRangeException's....
TRACE is lower than INFO so INFO level request logging would also be
recorded.

You can check for "Completed XXX request" in the log files to check the
request info with the correlation id.

Guozhang
On Mon, Dec 23, 2013 at 10:46 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

> Hmmm, it looks like I'm enabling all logging at INFO, and the request
> logging is only done at TRACE (why is that?).
>
> I suppose one wouldn't normally want to see request logs, so by default,
> they aren't enabled?
>
>
> On Mon, Dec 23, 2013 at 11:46 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Did you enable request log? It logs the ip of every request.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Mon, Dec 23, 2013 at 3:52 PM, Jason Rosenberg <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi Guozhang,
> > >
> > > I'm not sure I understand your first answer.  I don't see anything
> > > regarding the correlation id, elsewhere in the broker logs.....They
> only
> > > show up in those ERROR messages....
> > >
> > > I do see correlation id's in clients, but not on the broker.....
> > >
> > > Jason
> > >
> > >
> > > On Mon, Dec 23, 2013 at 6:46 PM, Guozhang Wang <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Jason,
> > > >
> > > > You can search the correlation id in the public access log on the
> > servers
> > > > to get the consumer information.
> > > >
> > > > As for logging, I agree that we should use the same level on both
> > sides.
> > > > Could you file a jira for this?
> > > >
> > > > Guozhang
> > > >
> > > >
> > > > On Mon, Dec 23, 2013 at 3:09 PM, Jason Rosenberg <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > In our broker logs, we occasionally see errors like this:
> > > > >
> > > > > 2013-12-23 05:02:08,456 ERROR [kafka-request-handler-2]
> > > server.KafkaApis
> > > > -
> > > > > [KafkaApi-45] Error when processing fetch request for partition
> > > > [mytopic,0]
> > > > > offset 204243601 from consumer with correlation id 130341
> > > > > kafka.common.OffsetOutOfRangeException: Request for offset
> 204243601
> > > but
> > > > we
> > > > > only have log segments in the range 204343397 to 207423640.
> > > > >
> > > > > I assume this means there's a consumer that has fallen behind
> > consuming
> > > > > messages, and the log retention policy has removed messages before
> > they
> > > > > could be consumed by the consumer.
> > > > >
> > > > > However, I'm not 100% which consumer it is, and it looks like the
> > only
> > > > info
> > > > > we have is the correlation id of the consumer, e.g.:
> > > > >
> > > > > "from consumer with correlation id 130341"
> > > > >
> > > > > Is there a way to know which consumer this refers to?  It seems
> there
> > > are
> > > > > far more correlation id's than there are consumers.  Would it be
> > > possible
> > > > > to provide a bit more descriptive error message here, so we can
> > > > immediately
> > > > > know which consumer is falling behind?
> > > > >
> > > > > We do see a corresponding entry in the consumer logs too:
> > > > >
> > > > > 2013-12-23 05:02:08,797  WARN
> > > > > [ConsumerFetcherThread-myconsumergroup-1387353494862-7aa0c61d-0-45]
> > > > > consumer.ConsumerFetcherThread -
> > > > >
> [ConsumerFetcherThread-myconsumergroup-1387353494862-7aa0c61d-0-45],
> > > > > Current offset 204243601 for partition [mytopic,0] out of range;
> > reset
> > > > > offset to 204343397
> > > > >
> > > > > But it would be nice to be able to also use the broker log to
> quickly
> > > > find
> > > > > consumers with issues.
> > > > >
> > > > > Also, I'm not sure, why is logging the event as an ERROR in the
> > broker,
> > > > but
> > > > > a WARN in the consumer?
> > > > >
> > > > > Jason
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > -- Guozhang
> > > >
> > >
> >
>

--
-- Guozhang