Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Hadoop Pi map-reduce job hangs


Copy link to this message
-
RE: Hadoop Pi map-reduce job hangs
Devaraj k 2012-08-08, 05:51
The location whatever you are looking is correct for map tasks. There might not be any exceptions in the map phase since it is completed 100%. It seems to be some problem in the reduce phase.

Can you check what is happening in the running reducer tasks logs(“/var/hadoop/tmp/mapred/local/userlogs/job_201208081600_0001/%Reduce-Task-ID%). Running reducer task ids can be found from from the Job Tracker UI.

Thanks

Devaraj

________________________________
From: Chandra Mohan, Ananda Vel Murugan [[EMAIL PROTECTED]]
Sent: Wednesday, August 08, 2012 11:02 AM
To: [EMAIL PROTECTED]
Subject: RE: Hadoop Pi map-reduce job hangs

Hi Devaraj,

Thanks for the response.

Do you mean the logs available in “/var/hadoop/tmp/mapred/local/userlogs/job_201208081600_0001/attempt_201208081600_0001_m_000005_0” folder?. I don’t see any log in stderr file.  Also syslog file has no exception or error.

Am I looking at correct file?

Regards,
Anand.C

________________________________
From: Devaraj k [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, August 08, 2012 10:45 AM
To: [EMAIL PROTECTED]
Subject: RE: Hadoop Pi map-reduce job hangs
Hi Anand,

   Can you check the task attempt logs for job, what is happening in the reduce task.

Thanks

Devaraj

________________________________
From: Chandra Mohan, Ananda Vel Murugan [[EMAIL PROTECTED]]
Sent: Wednesday, August 08, 2012 7:15 AM
To: [EMAIL PROTECTED]
Subject: Hadoop Pi map-reduce job hangs
Hi,

I have set up a three node cluster following the instructions here

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/

I ran into java.io.IOException: Incompatible namespaceIDs error and I fixed by manually updating the namespace ID. I restarted the cluster and tried running sample Pi calculation job. It hangs as shown below

[cid:[EMAIL PROTECTED]1660]

It is running for more than 30 minutes. I quickly checked the logs and did not find any exception. How do I fix this? Any help would be greatly appreciated.

Regards,
Anand.C