Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - More SocketTimeoutExceptions


+
Bob Jervis 2013-03-29, 22:35
Copy link to this message
-
Re: More SocketTimeoutExceptions
Jun Rao 2013-03-29, 23:09
This means that somehow the broker is not sending the response to the
producer and is sending it back to the wrong client. Any error in the
broker log?

Thanks,

Jun
On Fri, Mar 29, 2013 at 3:35 PM, Bob Jervis <[EMAIL PROTECTED]> wrote:

> I now have the following settings (in various configs):
>
> In our producer configs:
>
> producer.request.timeout.ms=600000
>
> This producer just hangs there for 10 minutes before timing out
>
> Here is the stack dump for that timeout:
>
> java.net.SocketTimeoutException
>         at
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:201)
>         at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)
>         at
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:221)
>         at kafka.utils.Utils$.read(Utils.scala:372)
>         at
>
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
>         at
> kafka.network.Receive$class.readCompletely(Transmission.scala:56)
>         at
>
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
>         at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
>         at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
>         at
>
> kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
>         at
>
> kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SyncProducer.scala:98)
>         at
>
> kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:98)
>         at
>
> kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:98)
>         at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
>         at
>
> kafka.producer.SyncProducer$$anonfun$send$1.apply$mcV$sp(SyncProducer.scala:97)
>         at
> kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:97)
>         at
> kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:97)
>         at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
>         at kafka.producer.SyncProducer.send(SyncProducer.scala:96)
>         at
>
> kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:221)
>         at
>
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:91)
>         at
>
> kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:85)
>         at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
>         at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
>         at scala.collection.Iterator$class.foreach(Iterator.scala:631)
>         at
> scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
>         at
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
>         at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
>         at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
>         at
>
> kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:85)
>         at
>
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:61)
>         at kafka.producer.Producer.send(Producer.scala:76)
>         at kafka.javaapi.producer.Producer.send(Producer.scala:41)
>         at
>
> com.visibletechnologies.platform.common.kafka.KafkaWriter.flush(KafkaWriter.java:131)
>         at
>
> com.visibletechnologies.platform.ingestion.ContentWriter.flushToKafka(ContentWriter.java:394)
>         at
>
> com.visibletechnologies.platform.ingestion.Midas.processPosts(Midas.java:430)
>         at
> com.visibletechnologies.platform.ingestion.Midas.doWork(Midas.java:194)
>         at
>
> com.visibletechnologies.framework.servicebase.ServiceBase.start(ServiceBase.java:187)
>         at