Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> [jira] [Commented] (KAFKA-1065) IO exception on windows when high throughput of messages


Copy link to this message
-
[jira] [Commented] (KAFKA-1065) IO exception on windows when high throughput of messages

    [ https://issues.apache.org/jira/browse/KAFKA-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776426#comment-13776426 ]

Jun Rao commented on KAFKA-1065:
--------------------------------

Could you try the patch in https://issues.apache.org/jira/browse/KAFKA-1008? I think it's fixing the same issue on windows.
                
> IO exception on windows when high throughput of messages
> --------------------------------------------------------
>
>                 Key: KAFKA-1065
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1065
>             Project: Kafka
>          Issue Type: Bug
>          Components: log
>    Affects Versions: 0.8
>         Environment: Windows 7 64bit
>            Reporter: Carl Austin
>            Assignee: Jun Rao
>
> When a large number of messages are sent per second to a broker on Windows a memory mapping exception occurs and kills Kafka. The exception follows :
> {code}kafka.common.KafkaStorageException: I/O exception in append to log 'test-0'
> at kafka.log.Log.append(Log.scala:349)
> at kafka.cluster.Partition.appendMessagesToLeader(Partition.scala:340)
> at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:236)
> at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:228)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
> at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
> at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
> at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
> at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
> at scala.collection.mutable.HashMap.map(HashMap.scala:39)
> at kafka.server.KafkaApis.appendToLocalLog(KafkaApis.scala:228)
> at kafka.server.KafkaApis.handleProducerRequest(KafkaApis.scala:162)
> at kafka.server.KafkaApis.handle(KafkaApis.scala:66)
> at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: The requested operation cannot be performed on a file with a user-mapped section open
> at java.io.RandomAccessFile.setLength(Native Method)
> at kafka.log.OffsetIndex.liftedTree2$1(OffsetIndex.scala:263)
> at kafka.log.OffsetIndex.resize(OffsetIndex.scala:262)
> at kafka.log.OffsetIndex.trimToValidSize(OffsetIndex.scala:247)
> at kafka.log.Log.rollToOffset(Log.scala:518)
> at kafka.log.Log.roll(Log.scala:502)
> at kafka.log.Log.maybeRoll(Log.scala:484)
> at kafka.log.Log.append(Log.scala:297)
> ... 19 more{code}
> This seems to have been mentioned in the past on the Kafka mail list and is an issue related to http://bugs.sun.com/view_bug.do?bug_id=4724038.
> Unfortunately this means that we cannot use Kafka on Windows.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB