Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Relationship between socket timeout & consumer wait timeout


Copy link to this message
-
Re: Relationship between socket timeout & consumer wait timeout

> case. Typically, SocketTimeout should be set to be at least maxWait or a
> bit larger.

The consumer-config documentation states that "The actual timeout set
will be max.fetch.wait + socket.timeout.ms." - however, that change
seems to have been lost in the code a while ago. I'll file a jira for
that - we should either fix the doc or re-introduce the addition.

Joel

On Tue, Nov 26, 2013 at 09:16:09AM -0800, Jun Rao wrote:
> Yes, it's a bit confusing. maxWait controls how long a fetch request will
> wait on the broker in the normal case. The issue is that if there is a hard
> crash on the broker (host is down), the client may not realize this
> immediately since TCP will try very hard to maintain the socket connection.
> By setting SocketTimeout, we allow the client to break out sooner in this
> case. Typically, SocketTimeout should be set to be at least maxWait or a
> bit larger.
>
> It's possible to specify an indefinite long poll by setting maxWait to a
> very large value. It's not recommended right now due to
> https://issues.apache.org/jira/browse/KAFKA-1016
>
> Thanks,
>
> Jun
>
>
> On Tue, Nov 26, 2013 at 3:43 AM, Simon Cooper <
> [EMAIL PROTECTED]> wrote:
>
> > I'm trying to use the SimpleConsumer to consume a queue, using long
> > polling. I'm encountering some difficulties when using very long poll times
> > (60s or more). It seems to be conflicting with the socket timeout.
> >
> > I thought the socket timeout was the timeout to actually connect to the
> > broker, then the wait timeout determined how long the long poll should be
> > for. This doesn't seem to be the case - the poll is throwing a
> > SocketTimeoutException even if it seems to have connected to the broker
> > successfully. Can anyone advise on this behaviour, and explain the
> > relationship between socket timeout and wait timeout?
> >
> > Also, is there any way of specifying an indefinite long poll for a
> > consumer?
> >
> > Thanks,
> > SimonC
> >
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB