Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> [jira] [Created] (KAFKA-1065) IO exception on windows when high throughput of messages


Copy link to this message
-
[jira] [Created] (KAFKA-1065) IO exception on windows when high throughput of messages
Carl Austin created KAFKA-1065:
----------------------------------

             Summary: IO exception on windows when high throughput of messages
                 Key: KAFKA-1065
                 URL: https://issues.apache.org/jira/browse/KAFKA-1065
             Project: Kafka
          Issue Type: Bug
          Components: producer
    Affects Versions: 0.8
         Environment: Windows 7 64bit
            Reporter: Carl Austin
            Assignee: Jun Rao
When a large number of messages are sent per second to a broker on Windows a memory mapping exception occurs and kills Kafka. The exception follows :

{code}kafka.common.KafkaStorageException: I/O exception in append to log 'test-0'
at kafka.log.Log.append(Log.scala:349)
at kafka.cluster.Partition.appendMessagesToLeader(Partition.scala:340)
at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:236)
at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:228)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
at scala.collection.Iterator$class.foreach(Iterator.scala:631)
at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
at scala.collection.mutable.HashMap.map(HashMap.scala:39)
at kafka.server.KafkaApis.appendToLocalLog(KafkaApis.scala:228)
at kafka.server.KafkaApis.handleProducerRequest(KafkaApis.scala:162)
at kafka.server.KafkaApis.handle(KafkaApis.scala:66)
at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: The requested operation cannot be performed on a file with a user-mapped section open
at java.io.RandomAccessFile.setLength(Native Method)
at kafka.log.OffsetIndex.liftedTree2$1(OffsetIndex.scala:263)
at kafka.log.OffsetIndex.resize(OffsetIndex.scala:262)
at kafka.log.OffsetIndex.trimToValidSize(OffsetIndex.scala:247)
at kafka.log.Log.rollToOffset(Log.scala:518)
at kafka.log.Log.roll(Log.scala:502)
at kafka.log.Log.maybeRoll(Log.scala:484)
at kafka.log.Log.append(Log.scala:297)
... 19 more{code}

This seems to have been mentioned in the past on the Kafka mail list and is an issue related to http://bugs.sun.com/view_bug.do?bug_id=4724038.

Unfortunately this means that we cannot use Kafka on Windows.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB