Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Relationship between socket timeout & consumer wait timeout


Copy link to this message
-
Re: Relationship between socket timeout & consumer wait timeout
Yes, it's a bit confusing. maxWait controls how long a fetch request will
wait on the broker in the normal case. The issue is that if there is a hard
crash on the broker (host is down), the client may not realize this
immediately since TCP will try very hard to maintain the socket connection.
By setting SocketTimeout, we allow the client to break out sooner in this
case. Typically, SocketTimeout should be set to be at least maxWait or a
bit larger.

It's possible to specify an indefinite long poll by setting maxWait to a
very large value. It's not recommended right now due to
https://issues.apache.org/jira/browse/KAFKA-1016

Thanks,

Jun
On Tue, Nov 26, 2013 at 3:43 AM, Simon Cooper <
[EMAIL PROTECTED]> wrote:

> I'm trying to use the SimpleConsumer to consume a queue, using long
> polling. I'm encountering some difficulties when using very long poll times
> (60s or more). It seems to be conflicting with the socket timeout.
>
> I thought the socket timeout was the timeout to actually connect to the
> broker, then the wait timeout determined how long the long poll should be
> for. This doesn't seem to be the case - the poll is throwing a
> SocketTimeoutException even if it seems to have connected to the broker
> successfully. Can anyone advise on this behaviour, and explain the
> relationship between socket timeout and wait timeout?
>
> Also, is there any way of specifying an indefinite long poll for a
> consumer?
>
> Thanks,
> SimonC
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB