Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Occasional batch send errors


Copy link to this message
-
Re: Occasional batch send errors
This means that the broker closed the socket connection for some reason.
The broker log around the same time should show the reason. Could you dig
that out?

Thanks,

Jun
On Tue, Apr 23, 2013 at 3:35 PM, Karl Kirch <[EMAIL PROTECTED]> wrote:

> I occasionally am getting some batch send errors from the stock async
> producer. This is on a cluster of 3 kafka (0.7.2) and 3 zookeeper nodes.
> Is there anyway to check what happens when those batch errors occur?
> Or bump up the retry count? (looks like it only did a single retry).
>
> I need the speed of the async producer, but it needs to be reliable (I see
> a handful of these a day but in a weather alerting system it only takes
> missing one let alone 25 or 100/1000).
>
> Here's a stack trace of one of the errors that I'm seeing.
>
> 22:23:39.405 [ProducerSendThread-1824508747] WARN
>  k.p.a.DefaultEventHandler - Error sending messages, 0 attempts remaining
> java.io.IOException: Connection reset by peer
>         at sun.nio.ch.FileDispatcher.writev0(Native Method) ~[na:1.6.0_24]
>         at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:33)
> ~[na:1.6.0_24]
>         at sun.nio.ch.IOUtil.write(IOUtil.java:125) ~[na:1.6.0_24]
>         at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:367)
> ~[na:1.6.0_24]
>         at java.nio.channels.SocketChannel.write(SocketChannel.java:360)
> ~[na:1.6.0_24]
>         at
> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:49)
> ~[apns-consumer-1.0.jar:na]
>         at kafka.network.Send$class.writeCompletely(Transmission.scala:73)
> ~[apns-consumer-1.0.jar:na]
>         at
> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
> ~[apns-consumer-1.0.jar:na]
>         at
> kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:95)
> ~[apns-consumer-1.0.jar:na]
>         at kafka.producer.SyncProducer.send(SyncProducer.scala:94)
> ~[apns-consumer-1.0.jar:na]
>         at kafka.producer.SyncProducer.multiSend(SyncProducer.scala:135)
> ~[apns-consumer-1.0.jar:na]
>         at
> kafka.producer.async.DefaultEventHandler.send(DefaultEventHandler.scala:58)
> [apns-consumer-1.0.jar:na]
>         at
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:44)
> [apns-consumer-1.0.jar:na]
>         at
> kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:116)
> [apns-consumer-1.0.jar:na]
>         at
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:95)
> [apns-consumer-1.0.jar:na]
>         at
> kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:71)
> [apns-consumer-1.0.jar:na]
>         at scala.collection.immutable.Stream.foreach(Stream.scala:260)
> [apns-consumer-1.0.jar:na]
>         at
> kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:70)
> [apns-consumer-1.0.jar:na]
>         at
> kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:41)
> [apns-consumer-1.0.jar:na]
> 22:23:39.406 [ProducerSendThread-1824508747] ERROR
> k.p.a.ProducerSendThread - Error in handling batch of 27 events
> java.io.IOException: Connection reset by peer
>         at sun.nio.ch.FileDispatcher.writev0(Native Method) ~[na:1.6.0_24]
>         at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:33)
> ~[na:1.6.0_24]
>         at sun.nio.ch.IOUtil.write(IOUtil.java:125) ~[na:1.6.0_24]
>         at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:367)
> ~[na:1.6.0_24]
>         at java.nio.channels.SocketChannel.write(SocketChannel.java:360)
> ~[na:1.6.0_24]
>         at
> kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:49)
> ~[apns-consumer-1.0.jar:na]
>         at kafka.network.Send$class.writeCompletely(Transmission.scala:73)
> ~[apns-consumer-1.0.jar:na]
>         at
> kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend.scala:25)
> ~[apns-consumer-1.0.jar:na]