Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> hadoop streaming job IOException


Copy link to this message
-
hadoop streaming job IOException
Hi, all

I got the following exception when I submit a hadoop streaming job to my
hadoop cluster.
I wrote the mapper in perl langguage, and there is no reducer. the mapper
script runs well on local server.
Hadoop version: Hadoop 0.20.2-CDH3B4
Can anyone give me some help?  What is the problem exactly?
java.io.IOException: log:null
R/W/S=500/0/0 in:NA [rec/s] out:NA [rec/s]
minRecWrittenToEnableSkip_=9223372036854775807 LOGNAME=null
HOST=null
USER=hadoop
HADOOP_USER=null
last Hadoop input: |null|
last tool output: |null|
Date: Mon Dec 19 09:49:47 CST 2011
java.io.IOException: Broken pipe
 at java.io.FileOutputStream.writeBytes(Native Method)
 at java.io.FileOutputStream.write(FileOutputStream.java:282)
 at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
 at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
 at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109)
 at java.io.DataOutputStream.write(DataOutputStream.java:90)
 at
org.apache.hadoop.streaming.io.TextInputWriter.writeUTF8(TextInputWriter.java:72)
 at
org.apache.hadoop.streaming.io.TextInputWriter.writeValue(TextInputWriter.java:51)
 at org.apache.hadoop.streaming.PipeMapper.map(PipeMapper.java:110)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
 at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:390)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:324)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
 at org.apache.hadoop.mapred.Child.main(Child.java:234)

 at org.apache.hadoop.streaming.PipeMapper.map(PipeMapper.java:126)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
 at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:390)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:324)
 at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:396)
 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
 at org.apache.hadoop.mapred.Child.main(Child.java:234)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB