Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Re: Socket timeouts in 0.8


Copy link to this message
-
Re: Socket timeouts in 0.8
Bob Jervis 2013-03-22, 16:38
I've tried this and it appears that we are still seeing the issue.  Here is
a stack trace of one of the socket timeout exceptions we are seeing (we
converted to the SimpleConsumer):

2013-03-22 04:54:51,807 INFO kafka.client.ClientUtils$: Fetching metadata
for topic Set(v1-japanese-0, v1-indonesian-5, v1-french-3, v1-other-4,
v1-portuguese-1, v1-other-1, v1-german-2, v1-english-15, v1-english-8,
v1-portuguese-6, v1-arabic-0, v1-english-6, v1-korean-1, v1-spanish-5,
v1-english-10, v1-japanese-4, v1-english-1, v1-italian-3, v1-spanish-1,
v1-english-5, v1-other-7, v1-portuguese-2, v1-other-0, v1-indonesian-4,
v1-english-9, v1-japanese-1, v1-spanish-2, v1-portuguese-7, v1-german-3,
v1-arabic-1, v1-japanese-7, v1-spanish-4, v1-other-3, v1-japanese-3,
v1-english-0, v1-french-1, v1-indonesian-0, v1-arabic-2, v1-english-4,
v1-french-0, v1-indonesian-7, v1-english-12, v1-spanish-3, v1-japanese-6,
v1-indonesian-3, v1-english-13, v1-other-6, v1-portuguese-3, v1-italian-1,
v1-english-7, v1-german-0, v1-korean-3, v1-spanish-7, v1-farsi-2,
v1-russian-1, v1-indonesian-1, v1-portuguese-0, v1-indonesian-6,
v1-french-2, v1-english-11, v1-japanese-2, v1-other-2, v1-indonesian-2,
v1-english-14, v1-portuguese-5, v1-spanish-6, v1-german-1, v1-italian-0,
v1-english-2, v1-japanese-5, v1-portuguese-4, v1-spanish-0, v1-english-3,
v1-arabic-3, v1-other-5, v1-korean-2)
2013-03-22 04:54:51,808 INFO kafka.producer.SyncProducer: Connected to
kafka01.qa.viq:9092 for producing
2013-03-22 04:54:51,983 INFO kafka.producer.SyncProducer: Disconnecting
from kafka01.qa.viq:9092
2013-03-22 04:54:51,987 INFO kafka.producer.SyncProducer: Connected to
100.100.106.91:9092 for producing
2013-03-22 14:41:42,650 INFO kafka.consumer.SimpleConsumer: Reconnect due
to socket error:
java.net.SocketTimeoutException
        at
sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:201)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)
        at
java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:221)
        at kafka.utils.Utils$.read(Utils.scala:372)
        at
kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
        at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
        at
kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
        at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
        at
kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:124)
        at
kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:122)
        at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:161)
        at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:161)
        at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:161)
        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
        at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:160)
        at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:160)
        at
kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:160)
        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
        at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:159)
        at
kafka.javaapi.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:48)
        at
kafka.javaapi.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:58)
        at
com.visibletechnologies.platform.common.kafka.KafkaReader.initializeIterator(KafkaReader.java:231)
        at
com.visibletechnologies.platform.common.kafka.KafkaReader.read(KafkaReader.java:181)
        at
com.visibletechnologies.platform.katta.krouter.KRouter.doWork(KRouter.java:117)
        at
com.visibletechnologies.framework.servicebase.ServiceBase.start(ServiceBase.java:187)
        at
com.visibletechnologies.platform.katta.krouter.Main.main(Main.java:132)

I don't see any Zookeeper interactions here.  It looks like fetch requests
to the Kafka broker, but maybe I'm missing something.
On Thu, Mar 21, 2013 at 9:16 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
 
+
Jun Rao 2013-03-22, 17:08
+
Bob Jervis 2013-03-22, 18:01
+
Bob Jervis 2013-03-22, 18:28
+
Neha Narkhede 2013-03-22, 19:54
+
Bob Jervis 2013-03-22, 23:25
+
Bob Jervis 2013-03-22, 16:44