Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - help!!!,what is happened with my project?


Copy link to this message
-
RE: help!!!,what is happened with my project?
Devaraj k 2013-09-11, 09:34
It seems filesystem is closed in the Task. Do you see any error in Datanode/Namenode(HDFS) log files?

Thanks
Devaraj k

From: heyamin [mailto:[EMAIL PROTECTED]]
Sent: 11 September 2013 14:28
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: help!!!,what is happened with my project?
Hi:

        Today when I run a task,I get some warnings,what is happened?

2013-09-11 16:45:17,486 INFO org.Apache.adopt.util.NativeCodeLoader: Loaded the native-hadoop library

2013-09-11 16:45:18,680 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0

2013-09-11 16:45:18,708 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2ddc7394<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2ddc7394>

2013-09-11 16:45:18,998 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://192.168.1.240:9000/user/hadoop/input/20130815-log.log:0+118673837

2013-09-11 16:45:19,027 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 40

2013-09-11 16:45:19,113 INFO org.apache.hadoop.mapred.MapTask: data buffer = 31876710/39845888

2013-09-11 16:45:19,113 INFO org.apache.hadoop.mapred.MapTask: record buffer = 104857/131072

2013-09-11 16:45:19,136 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library not loaded

2013-09-11 16:45:23,374 INFO org.apache.hadoop.mapred.MapTask: Spilling map output: record full = true

2013-09-11 16:45:23,374 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 3082708; bufvoid = 39845888

2013-09-11 16:45:23,374 INFO org.apache.hadoop.mapred.MapTask: kvstart = 0; kvend = 104857; length = 131072

2013-09-11 16:45:23,661 INFO org.apache.hadoop.mapred.MapTask: Ignoring exception during close for org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader@36781b93<mailto:org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader@36781b93>

java.io.IOException: Filesystem closed

      at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:323)

      at org.apache.hadoop.hdfs.DFSClient.access$1200(DFSClient.java:78)

      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.close(DFSClient.java:2326)

      at java.io.FilterInputStream.close(FilterInputStream.java:181)

      at org.apache.hadoop.util.LineReader.close(LineReader.java:145)

      at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.close(LineRecordReader.java:187)

      at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.close(MapTask.java:496)

      at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:1776)

      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:778)

      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)

      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

      at java.security.AccessController.doPrivileged(Native Method)

      at javax.security.auth.Subject.doAs(Subject.java:415)

      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

      at org.apache.hadoop.mapred.Child.main(Child.java:249)

2013-09-11 16:45:23,662 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output