Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: Creation of symlink from failed


Copy link to this message
-
Re: Creation of symlink from failed
Pradeep Singh 2013-08-28, 06:47
Hi All ,
I  Guess the problem might has something to do with the hadoop setting and
permissions related to Symlink creation .
*Creation of symlink from
/windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
to
/tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
failed\.*

Can anybody throw some light on this .

Regards
Pradeep Singh
On Wed, Aug 28, 2013 at 12:37 AM, Pradeep Singh <[EMAIL PROTECTED]>wrote:

> Hi All ,
> I am trying to run a simple steaming command as mentioned below .
>
>  bin/hadoop jar
> /windows/Hadoop/hadoop-1.2.1/contrib/streaming/hadoop-streaming-1.2.1.jar
> -input /usr/pradeep/input/'Good words' -output /usr/pradeep/output mapper
> /bin/cat reducer /bin/wc -w
>
> I am getting below as message in the error file
>
> packageJobJar: [/tmp/hadoop-root/hadoop-unjar3249316336279467583/] []
> /tmp/streamjob1400561611088699684.jar tmpDir=null
> 13/08/27 23:55:54 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 13/08/27 23:55:54 WARN snappy.LoadSnappy: Snappy native library not loaded
> 13/08/27 23:55:54 INFO mapred.FileInputFormat: Total input paths to
> process : 1
> 13/08/27 23:55:55 INFO streaming.StreamJob: getLocalDirs():
> [/tmp/hadoop-root/mapred/local]
> 13/08/27 23:55:55 INFO streaming.StreamJob: Running job:
> job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:55:55 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:55:55 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:55:56 INFO streaming.StreamJob:  map 0%  reduce 0%
> 13/08/27 23:56:21 INFO streaming.StreamJob:  map 100%  reduce 100%
> 13/08/27 23:56:21 INFO streaming.StreamJob: To kill this job, run:
> 13/08/27 23:56:21 INFO streaming.StreamJob:
> /windows/Hadoop/hadoop-1.2.1/libexec/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201308272335_0004
> 13/08/27 23:56:21 INFO streaming.StreamJob: Tracking URL:
> http://localhost:50030/jobdetails.jsp?jobid=job_201308272335_0004
> 13/08/27 23:56:21 ERROR streaming.StreamJob: Job not successful. Error:
> JobCleanup Task Failure, Task: task_201308272335_0004_m_000002
> 13/08/27 23:56:21 INFO streaming.StreamJob: killJob...
> Streaming Command Failed!
>
> I checked the log and this is what i found :
> MapAttempt TASK_TYPE="CLEANUP" TASKID="task_201308272335_0004_m_000002"
> TASK_ATTEMPT_ID="attempt_201308272335_0004_m_000002_0" TASK_STATUS="FAILED"
> FINISH_TIME="1377627971824" HOSTNAME="pradeep-pc"
> ERROR="java\.lang\.Throwable: Child Error
>     at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:271)
> Caused by: java\.io\.IOException: Creation of symlink from
> /windows/Hadoop/hadoop-1\.2\.1/libexec/\.\./logs/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> to
> /tmp/hadoop-root/mapred/local/userlogs/job_201308272335_0004/attempt_201308272335_0004_m_000002_0
> failed\.
>     at
> org\.apache\.hadoop\.mapred\.TaskLog\.createTaskAttemptLogDir(TaskLog\.java:110)
>     at
> org\.apache\.hadoop\.mapred\.DefaultTaskController\.createLogDir(DefaultTaskController\.java:71)
>     at
> org\.apache\.hadoop\.mapred\.TaskRunner\.prepareLogFiles(TaskRunner\.java:316)
>     at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:228)
>
> Cant understand the reason for such error .Everything seems to be fine
> .Would appreicate any help in this respect .
>
> Regards
> Pradeep Singh
>