Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> help!!!,what is happened with my project?


Copy link to this message
-
RE: help!!!,what is happened with my project?
It seems filesystem is closed in the Task. Do you see any error in Datanode/Namenode(HDFS) log files?

Thanks
Devaraj k

From: heyamin [mailto:[EMAIL PROTECTED]]
Sent: 11 September 2013 14:28
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: help!!!,what is happened with my project?
Hi:

        Today when I run a task,I get some warnings,what is happened?

2013-09-11 16:45:17,486 INFO org.Apache.adopt.util.NativeCodeLoader: Loaded the native-hadoop library

2013-09-11 16:45:18,680 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0

2013-09-11 16:45:18,708 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2ddc7394<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2ddc7394>

2013-09-11 16:45:18,998 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://192.168.1.240:9000/user/hadoop/input/20130815-log.log:0+118673837

2013-09-11 16:45:19,027 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 40

2013-09-11 16:45:19,113 INFO org.apache.hadoop.mapred.MapTask: data buffer = 31876710/39845888

2013-09-11 16:45:19,113 INFO org.apache.hadoop.mapred.MapTask: record buffer = 104857/131072

2013-09-11 16:45:19,136 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library not loaded

2013-09-11 16:45:23,374 INFO org.apache.hadoop.mapred.MapTask: Spilling map output: record full = true

2013-09-11 16:45:23,374 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 3082708; bufvoid = 39845888

2013-09-11 16:45:23,374 INFO org.apache.hadoop.mapred.MapTask: kvstart = 0; kvend = 104857; length = 131072

2013-09-11 16:45:23,661 INFO org.apache.hadoop.mapred.MapTask: Ignoring exception during close for org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader@36781b93<mailto:org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader@36781b93>

java.io.IOException: Filesystem closed

      at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:323)

      at org.apache.hadoop.hdfs.DFSClient.access$1200(DFSClient.java:78)

      at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.close(DFSClient.java:2326)

      at java.io.FilterInputStream.close(FilterInputStream.java:181)

      at org.apache.hadoop.util.LineReader.close(LineReader.java:145)

      at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.close(LineRecordReader.java:187)

      at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.close(MapTask.java:496)

      at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:1776)

      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:778)

      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)

      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

      at java.security.AccessController.doPrivileged(Native Method)

      at javax.security.auth.Subject.doAs(Subject.java:415)

      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

      at org.apache.hadoop.mapred.Child.main(Child.java:249)

2013-09-11 16:45:23,662 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB