Kafka, mail # user - Re: Consumer in the mirror maker stop fetching logs - 2013-05-04, 11:58
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Consumer in the mirror maker stop fetching logs
Thanks Jun.

I grep the server log, and found  3 kinds of exception on broker.

1.
kafka.common.OffsetOutOfRangeException

I don't think this could cause broker closing the socket.
2.
579740110 [kafka-processor-6] ERROR kafka.network.Processor  - Closing
socket for /10.47.7.56 because of error
java.io.IOException: Broken pipe
        at sun.nio.ch.FileChannelImpl.transferTo0(Native Method)
        at
sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:405)
        at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:506)
        at kafka.message.FileMessageSet.writeTo(FileMessageSet.scala:102)
        at kafka.server.MessageSetSend.writeTo(MessageSetSend.scala:53)
        at kafka.network.MultiSend.writeTo(Transmission.scala:91)
        at kafka.network.Processor.write(SocketServer.scala:339)
        at kafka.network.Processor.run(SocketServer.scala:216)
        at java.lang.Thread.run(Thread.java:662)

3.
580861724 [kafka-processor-13] ERROR kafka.network.Processor  - Closing
socket for /10.47.7.56 because of error
java.nio.channels.ClosedChannelException
        at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:88)
        at sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:486)
        at kafka.message.FileMessageSet.writeTo(FileMessageSet.scala:102)
        at kafka.server.MessageSetSend.writeTo(MessageSetSend.scala:53)
        at kafka.network.MultiSend.writeTo(Transmission.scala:91)
        at kafka.network.Processor.write(SocketServer.scala:339)
        at kafka.network.Processor.run(SocketServer.scala:216)
        at java.lang.Thread.run(Thread.java:662)

probably the second exception may be the cause.

also, I have found the EOFException on consumer side doesn't cause stopping
fetching every time.
On Fri, May 3, 2013 at 11:50 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
*Best Regards

向河林*

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB