Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Connection reset by peer


+
Yonghui Zhao 2013-03-18, 03:25
Copy link to this message
-
Re: Connection reset by peer
Jun Rao 2013-03-18, 16:25
The error you saw on the broker is for consumer requests, not for producer.
For the issues in the producer, are you using a VIP? Is there any firewall
btw producer and broker? The typical issues with "connection reset" that we
have seen are caused by the load balancer or the firewall killing idle
connections.

Thanks,

Jun

On Sun, Mar 17, 2013 at 8:24 PM, Yonghui Zhao <[EMAIL PROTECTED]> wrote:

> In kafka 0.7.2, I use a producer to send 200 million message to kafka
> server.
>
> After sent 100 million this exception happend:
>
> In producer:
>
> Exception in thread "main" java.io.IOException: Connection reset by peer
>     at sun.nio.ch.FileDispatcher.writev0(Native Method)
>     at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51)
>     at sun.nio.ch.IOUtil.write(IOUtil.java:182)
>     at sun.nio.ch.SocketChannelImpl.write0(SocketChannelImpl.java:383)
>     at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:406)
>     at java.nio.channels.SocketChannel.write(SocketChannel.java:384)
>     at
> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:49)
>     at kafka.network.Send$class.writeCompletely(Transmission.scala:73)
>     at
>
> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
>     at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:95)
>     at kafka.producer.SyncProducer.send(SyncProducer.scala:94)
>     at kafka.producer.SyncProducer.send(SyncProducer.scala:125)
>     at
>
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:114)
>     at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
>     at
> kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
>     at
>
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:57)
>     at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:43)
>     at kafka.producer.ProducerPool.send(ProducerPool.scala:100)
>     at kafka.producer.Producer.zkSend(Producer.scala:137)
>     at kafka.producer.Producer.send(Producer.scala:99)
>     at kafka.javaapi.producer.Producer.send(Producer.scala:103)
>
>
> In kafka server:
>
> [2013-03-16 06:59:49,491] ERROR Closing socket for /10.2.201.201 because
> of
> error (kafka.network.Processor)
> java.io.IOException: Connection reset by peer
>     at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
>     at
> sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:456)
>     at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:557)
>     at kafka.message.FileMessageSet.writeTo(FileMessageSet.scala:102)
>     at kafka.server.MessageSetSend.writeTo(MessageSetSend.scala:53)
>     at kafka.network.MultiSend.writeTo(Transmission.scala:91)
>     at kafka.network.Processor.write(SocketServer.scala:339)
>     at kafka.network.Processor.run(SocketServer.scala:216)
>     at java.lang.Thread.run(Thread.java:679)
>
> Have you ever seen this exception before, what's the root cause? Thanks
>

 
+
Yonghui Zhao 2013-03-19, 07:29
+
Yonghui Zhao 2013-03-19, 08:35
+
Jun Rao 2013-03-19, 16:15
+
Yonghui Zhao 2013-03-21, 04:26
+
Jun Rao 2013-03-22, 15:06
+
Yonghui Zhao 2013-03-22, 15:46
+
Yonghui Zhao 2013-03-25, 06:14
+
Neha Narkhede 2013-03-25, 13:49
+
Yonghui Zhao 2013-03-26, 02:14
+
Neha Narkhede 2013-03-26, 03:16
+
Yonghui Zhao 2013-03-26, 03:49
+
Neha Narkhede 2013-03-26, 15:40
+
Yonghui Zhao 2013-03-26, 16:48
+
Neha Narkhede 2013-03-27, 14:52
+
Yonghui Zhao 2013-03-28, 03:35
+
Jun Rao 2013-03-28, 04:54
+
Yonghui Zhao 2013-03-28, 07:23
+
Jun Rao 2013-03-28, 14:52
+
Yonghui Zhao 2013-03-28, 15:21
+
Jun Rao 2013-03-28, 15:25
+
Yonghui Zhao 2013-03-28, 15:32
+
Jun Rao 2013-03-29, 04:03
+
Ranjith Venkatesan 2014-02-02, 06:03