Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - hadoop streaming job IOException


Copy link to this message
-
hadoop streaming job IOException
Yu Yang 2011-12-19, 03:45
Hi, all

I got the following exception when I submit a hadoop streaming job to my
hadoop cluster.
I wrote the mapper in perl langguage, and there is no reducer. the mapper
script runs well on local server.
Hadoop version: Hadoop 0.20.2-CDH3B4
Can anyone give me some help?  What is the problem exactly?
java.io.IOException: log:null
R/W/S=500/0/0 in:NA [rec/s] out:NA [rec/s]
minRecWrittenToEnableSkip_=9223372036854775807 LOGNAME=null
HOST=null
USER=hadoop
HADOOP_USER=null
last Hadoop input: |null|
last tool output: |null|
Date: Mon Dec 19 09:49:47 CST 2011
java.io.IOException: Broken pipe
 at java.io.FileOutputStream.writeBytes(Native Method)
 at java.io.FileOutputStream.write(FileOutputStream.java:282)
 at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
 at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
 at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
 at java.io.DataOutputStream.write(DataOutputStream.java:90)
 at
org.apache.hadoop.streaming.io.TextInputWriter.writeUTF8(TextInputWriter.java:72)
 at
org.apache.hadoop.streaming.io.TextInputWriter.writeValue(TextInputWriter.java:51)
 at org.apache.hadoop.streaming.PipeMapper.map(PipeMapper.java:110)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
 at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:390)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:324)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
 at org.apache.hadoop.mapred.Child.main(Child.java:234)

 at org.apache.hadoop.streaming.PipeMapper.map(PipeMapper.java:126)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
 at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:390)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:324)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
 at org.apache.hadoop.mapred.Child.main(Child.java:234)