Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> terasort fails with error


Copy link to this message
-
Re: terasort fails with error

It seems like some of your nodes don't have correct permissions on the hadoop-log-dir, you should check that. It may be only a subset of nodes as you mentioned that Teragen seems to work.

You can start with the nodes where the tasks were reported as failed.

HTH,
+Vinod

On Aug 30, 2012, at 10:08 AM, Jain, Prem wrote:

> Folks,
>  
> I am trying to terasort and seems to fail with the following error ?  Teragen seems to work well though
>  
> 12/08/30 09:53:38 WARN mapred.JobClient: Error reading taskoutputhttp://datanode004.svl.ibm.com:50060/tasklog?plaintext=true&attemptid=attempt_201208300948_0001_m_000060_1&filter=stdout
> 12/08/30 09:53:38 WARN mapred.JobClient: Error reading task outputhttp:// datanode004.svl.ibm.com:50060/tasklog?plaintext=true&attemptid=attempt_201208300948_0001_m_000060_1&filter=stderr
> 12/08/30 09:53:39 INFO mapred.JobClient: Task Id : attempt_201208300948_0001_m_000152_1, Status : FAILED
> ENOENT: No such file or directory
>         at org.apache.hadoop.io.nativeio.NativeIO.open(Native Method)
>         at org.apache.hadoop.io.SecureIOUtils.createForWrite(SecureIOUtils.java:161)
>         at org.apache.hadoop.mapred.TaskLog.writeToIndexFile(TaskLog.java:296)
>         at org.apache.hadoop.mapred.TaskLog.syncLogs(TaskLog.java:369)
>         at org.apache.hadoop.mapred.Child.main(Child.java:229
>  
> The command I am running is:
> $ hadoop jar $HADOOP_HOME/hadoop-examples-1.0.0.jar terasort /user/netapp/input-dir /user/netapp/output-dir
>  
>  
> Thanks in advance
> Prem
>  

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB