Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> what does it mean -- java.io.IOException: Filesystem closed


Copy link to this message
-
what does it mean -- java.io.IOException: Filesystem closed
Hi ,
  Running jadoop job from time to time I got such exception (from one of the
reducers):

The questions are :
1) What does this exception means for the data integrity?
2) Does it mean that part of the data which reducer responsible for (and got
exception) are lost?
3) What could cause for such exception?

  java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:222)
at org.apache.hadoop.hdfs.DFSClient.access$600(DFSClient.java:66)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.writeChunk(DFSClient.java:2948)
at
org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:150)
at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:132)
at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:121)
at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:112)
at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:49)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at
org.apache.hadoop.mapreduce.lib.output.TextOutputFormat$LineRecordWriter.writeObject(TextOutputFormat.java:78)
at
org.apache.hadoop.mapreduce.lib.output.TextOutputFormat$LineRecordWriter.write(TextOutputFormat.java:99)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at
com.analytics.hbase.internals.MergeMapReduceHDFSInserter$MergeMapReduceHDFSInserterReducer.reduce(Unknown
Source)
at
com.analytics.hbase.internals.MergeMapReduceHDFSInserter$MergeMapReduceHDFSInserterReducer.reduce(Unknown
Source)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:563)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)

Thanks in Advance
Oleg.
+
Jianwu Wang 2010-11-02, 23:43
+
Sudhir Vallamkondu 2010-11-04, 04:58