Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Connection reset by peer


Copy link to this message
-
Re: Connection reset by peer
Thanks Jun.

Now I use onebox to test kafka, kafka server ip on zk is 127.0.0.1, network
is not affected by external factors.

Reset connection is not reproed, but I still find Broken pipe exceptions
and a few zk exceptions.

[2013-03-19 15:23:28,660] INFO Closed socket connection for client /
127.0.0.1:51902 which had sessionid 0x13d8152007b002c
(org.apache.zookeeper.server.NIOServerCnxn)
[2013-03-19 15:23:28,672] ERROR Unexpected Exception:
(org.apache.zookeeper.server.NIOServerCnxn)
java.nio.channels.CancelledKeyException
    at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
    at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
    at
org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:418)
    at
org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1509)
    at
org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:171)
    at
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:135)

[2013-03-19 15:15:58,355] INFO Closing socket connection to /127.0.0.1.
(kafka.network.Processor)
[2013-03-19 15:16:00,161] INFO Closing socket connection to /127.0.0.1.
(kafka.network.Processor)
[2013-03-19 15:16:01,784] INFO Closing socket connection to /127.0.0.1.
(kafka.network.Processor)
[2013-03-19 15:16:04,751] INFO Closing socket connection to /127.0.0.1.
(kafka.network.Processor)
[2013-03-19 15:16:07,734] ERROR Closing socket for /127.0.0.1 because of
error (kafka.network.Processor)
java.io.IOException: Broken pipe
    at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
    at
sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:456)
    at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:557)
    at kafka.message.FileMessageSet.writeTo(FileMessageSet.scala:102)
    at kafka.server.MessageSetSend.writeTo(MessageSetSend.scala:53)
    at kafka.network.MultiSend.writeTo(Transmission.scala:91)
    at kafka.network.Processor.write(SocketServer.scala:339)
    at kafka.network.Processor.run(SocketServer.scala:216)
    at java.lang.Thread.run(Thread.java:679)
2013/3/19 Jun Rao <[EMAIL PROTECTED]>

> The error you saw on the broker is for consumer requests, not for producer.
> For the issues in the producer, are you using a VIP? Is there any firewall
> btw producer and broker? The typical issues with "connection reset" that we
> have seen are caused by the load balancer or the firewall killing idle
> connections.
>
> Thanks,
>
> Jun
>
> On Sun, Mar 17, 2013 at 8:24 PM, Yonghui Zhao <[EMAIL PROTECTED]>
> wrote:
>
> > In kafka 0.7.2, I use a producer to send 200 million message to kafka
> > server.
> >
> > After sent 100 million this exception happend:
> >
> > In producer:
> >
> > Exception in thread "main" java.io.IOException: Connection reset by peer
> >     at sun.nio.ch.FileDispatcher.writev0(Native Method)
> >     at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51)
> >     at sun.nio.ch.IOUtil.write(IOUtil.java:182)
> >     at sun.nio.ch.SocketChannelImpl.write0(SocketChannelImpl.java:383)
> >     at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:406)
> >     at java.nio.channels.SocketChannel.write(SocketChannel.java:384)
> >     at
> >
> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:49)
> >     at kafka.network.Send$class.writeCompletely(Transmission.scala:73)
> >     at
> >
> >
> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
> >     at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:95)
> >     at kafka.producer.SyncProducer.send(SyncProducer.scala:94)
> >     at kafka.producer.SyncProducer.send(SyncProducer.scala:125)
> >     at
> >
> >
> kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.scala:114)
> >     at
> > kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)
> >     at
> > kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:100)

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB