Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Relationship between socket timeout & consumer wait timeout


Copy link to this message
-
Re: Relationship between socket timeout & consumer wait timeout

> case. Typically, SocketTimeout should be set to be at least maxWait or a
> bit larger.

The consumer-config documentation states that "The actual timeout set
will be max.fetch.wait + socket.timeout.ms." - however, that change
seems to have been lost in the code a while ago. I'll file a jira for
that - we should either fix the doc or re-introduce the addition.

Joel

On Tue, Nov 26, 2013 at 09:16:09AM -0800, Jun Rao wrote:
> Yes, it's a bit confusing. maxWait controls how long a fetch request will
> wait on the broker in the normal case. The issue is that if there is a hard
> crash on the broker (host is down), the client may not realize this
> immediately since TCP will try very hard to maintain the socket connection.
> By setting SocketTimeout, we allow the client to break out sooner in this
> case. Typically, SocketTimeout should be set to be at least maxWait or a
> bit larger.
>
> It's possible to specify an indefinite long poll by setting maxWait to a
> very large value. It's not recommended right now due to
> https://issues.apache.org/jira/browse/KAFKA-1016
>
> Thanks,
>
> Jun
>
>
> On Tue, Nov 26, 2013 at 3:43 AM, Simon Cooper <
> [EMAIL PROTECTED]> wrote:
>
> > I'm trying to use the SimpleConsumer to consume a queue, using long
> > polling. I'm encountering some difficulties when using very long poll times
> > (60s or more). It seems to be conflicting with the socket timeout.
> >
> > I thought the socket timeout was the timeout to actually connect to the
> > broker, then the wait timeout determined how long the long poll should be
> > for. This doesn't seem to be the case - the poll is throwing a
> > SocketTimeoutException even if it seems to have connected to the broker
> > successfully. Can anyone advise on this behaviour, and explain the
> > relationship between socket timeout and wait timeout?
> >
> > Also, is there any way of specifying an indefinite long poll for a
> > consumer?
> >
> > Thanks,
> > SimonC
> >