Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Connection reset by peer


Copy link to this message
-
Re: Connection reset by peer
Thanks Jun.

Now I use onebox to test kafka, kafka server ip on zk is 127.0.0.1, network
is not affected by external factors.

Reset connection is not reproed, but I still find Broken pipe exceptions
and a few zk exceptions.

[2013-03-19 15:23:28,660] INFO Closed socket connection for client /
127.0.0.1:51902 which had sessionid 0x13d8152007b002c
(org.apache.zookeeper.server.NIOServerCnxn)
[2013-03-19 15:23:28,672] ERROR Unexpected Exception:
(org.apache.zookeeper.server.NIOServerCnxn)
java.nio.channels.CancelledKeyException
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
    at
org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:418)
    at
org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1509)
    at
org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:171)
    at
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:135)

[2013-03-19 15:15:58,355] INFO Closing socket connection to /127.0.0.1.
(kafka.network.Processor)
[2013-03-19 15:16:00,161] INFO Closing socket connection to /127.0.0.1.
(kafka.network.Processor)
[2013-03-19 15:16:01,784] INFO Closing socket connection to /127.0.0.1.
(kafka.network.Processor)
[2013-03-19 15:16:04,751] INFO Closing socket connection to /127.0.0.1.
(kafka.network.Processor)
[2013-03-19 15:16:07,734] ERROR Closing socket for /127.0.0.1 because of
error (kafka.network.Processor)
java.io.IOException: Broken pipe
    at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
    at
sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:456)
    at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:557)
    at kafka.message.FileMessageSet.writeTo(FileMessageSet.scala:102)
    at kafka.server.MessageSetSend.writeTo(MessageSetSend.scala:53)
    at kafka.network.MultiSend.writeTo(Transmission.scala:91)
    at kafka.network.Processor.write(SocketServer.scala:339)
    at kafka.network.Processor.run(SocketServer.scala:216)
    at java.lang.Thread.run(Thread.java:679)
2013/3/19 Jun Rao <[EMAIL PROTECTED]>

> The error you saw on the broker is for consumer requests, not for producer.
> For the issues in the producer, are you using a VIP? Is there any firewall
> btw producer and broker? The typical issues with "connection reset" that we
> have seen are caused by the load balancer or the firewall killing idle
> connections.
>
> Thanks,
>
> Jun
>
> On Sun, Mar 17, 2013 at 8:24 PM, Yonghui Zhao <[EMAIL PROTECTED]>
> wrote:
>
> > In kafka 0.7.2, I use a producer to send 200 million message to kafka
> > server.
> >
> > After sent 100 million this exception happend:
> >
> > In producer:
> >
> > Exception in thread "main" java.io.IOException: Connection reset by peer
> >     at sun.nio.ch.FileDispatcher.writev0(Native Method)
> >     at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51)
> >     at sun.nio.ch.IOUtil.write(IOUtil.java:182)
> >     at sun.nio.ch.SocketChannelImpl.write0(SocketChannelImpl.java:383)
> >     at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:406)
> >     at java.nio.channels.SocketChannel.write(SocketChannel.java:384)
> >     at
> >
> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:49)
> >     at kafka.network.Send$class.writeCompletely(Transmission.scala:73)
> >     at
> >
> >
> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
> >     at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:95)
> >     at kafka.producer.SyncProducer.send(SyncProducer.scala:94)
> >     at kafka.producer.SyncProducer.send(SyncProducer.scala:125)
> >     at
> >
> >
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:114)
> >     at
> > kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
> >     at
> > kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)