Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Re: FileSystem Closed.
Aniket Mokashi
2012-07-11, 16:42
안의건
2012-07-11, 00:09
Copy link to this message
-
Re: FileSystem Closed.
Harsh J 2012-07-10, 16:39
This appears to be a Hive issue (something probably called FS.close()
too early?). Redirecting to the Hive user lists as they can help
better with this.

On Tue, Jul 10, 2012 at 9:59 PM, 안의건 <[EMAIL PROTECTED]> wrote:
> Hello. I have a problem with the filesystem closing.
>
> The filesystem was closed when the hive query is running.
> It is 'select' query and the data size is about 1TB.
> I'm using hadoop-0.20.2 and hive-0.7.1.
>
> The error log is telling that tmp file is not deleted, or the tmp path
> exception is occurred.
>
> Is there any hadoop configuration I'm missing?
>
> Thank you
>
> [stderr logs]
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> Filesystem closed
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:454)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:636)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:557)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:193)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:226)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:617)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:453)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648)
> at org.apache.hadoop.fs.FileSystem.deleteOnExit(FileSystem.java:615)
> at
> org.apache.hadoop.hive.shims.Hadoop20Shims.fileSystemDeleteOnExit(Hadoop20Shims.java:68)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.createBucketFiles(FileSinkOperator.java:451)
> ... 12 more

--
Harsh J
+
Aniket Mokashi 2012-07-11, 16:42
+
안의건 2012-07-11, 00:09