Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - terasort fails with error


+
Jain, Prem 2012-08-30, 17:08
Copy link to this message
-
Re: terasort fails with error
Vinod Kumar Vavilapalli 2012-08-30, 18:10

It seems like some of your nodes don't have correct permissions on the hadoop-log-dir, you should check that. It may be only a subset of nodes as you mentioned that Teragen seems to work.

You can start with the nodes where the tasks were reported as failed.

HTH,
+Vinod

On Aug 30, 2012, at 10:08 AM, Jain, Prem wrote:

> Folks,
>  
> I am trying to terasort and seems to fail with the following error ?  Teragen seems to work well though
>  
> 12/08/30 09:53:38 WARN mapred.JobClient: Error reading taskoutputhttp://datanode004.svl.ibm.com:50060/tasklog?plaintext=true&attemptid=attempt_201208300948_0001_m_000060_1&filter=stdout
> 12/08/30 09:53:38 WARN mapred.JobClient: Error reading task outputhttp:// datanode004.svl.ibm.com:50060/tasklog?plaintext=true&attemptid=attempt_201208300948_0001_m_000060_1&filter=stderr
> 12/08/30 09:53:39 INFO mapred.JobClient: Task Id : attempt_201208300948_0001_m_000152_1, Status : FAILED
> ENOENT: No such file or directory
>         at org.apache.hadoop.io.nativeio.NativeIO.open(Native Method)
>         at org.apache.hadoop.io.SecureIOUtils.createForWrite(SecureIOUtils.java:161)
>         at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:296)
>         at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:369)
>         at org.apache.hadoop.mapred.Child.main(Child.java:229
>  
> The command I am running is:
> $ hadoop jar $HADOOP_HOME/hadoop-examples-1.0.0.jar terasort /user/netapp/input-dir /user/netapp/output-dir
>  
>  
> Thanks in advance
> Prem
>  

+
Jain, Prem 2012-08-30, 19:31