Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Hadoop Pi map-reduce job hangs


Copy link to this message
-
RE: Hadoop Pi map-reduce job hangs
The location whatever you are looking is correct for map tasks. There might not be any exceptions in the map phase since it is completed 100%. It seems to be some problem in the reduce phase.

Can you check what is happening in the running reducer tasks logs(“/var/hadoop/tmp/mapred/local/userlogs/job_201208081600_0001/%Reduce-Task-ID%). Running reducer task ids can be found from from the Job Tracker UI.

Thanks

Devaraj

________________________________
From: Chandra Mohan, Ananda Vel Murugan [[EMAIL PROTECTED]]
Sent: Wednesday, August 08, 2012 11:02 AM
To: [EMAIL PROTECTED]
Subject: RE: Hadoop Pi map-reduce job hangs

Hi Devaraj,

Thanks for the response.

Do you mean the logs available in “/var/hadoop/tmp/mapred/local/userlogs/job_201208081600_0001/attempt_201208081600_0001_m_000005_0” folder?. I don’t see any log in stderr file.  Also syslog file has no exception or error.

Am I looking at correct file?

Regards,
Anand.C

________________________________
From: Devaraj k [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, August 08, 2012 10:45 AM
To: [EMAIL PROTECTED]
Subject: RE: Hadoop Pi map-reduce job hangs
Hi Anand,

   Can you check the task attempt logs for job, what is happening in the reduce task.

Thanks

Devaraj

________________________________
From: Chandra Mohan, Ananda Vel Murugan [[EMAIL PROTECTED]]
Sent: Wednesday, August 08, 2012 7:15 AM
To: [EMAIL PROTECTED]
Subject: Hadoop Pi map-reduce job hangs
Hi,

I have set up a three node cluster following the instructions here

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/

I ran into java.io.IOException: Incompatible namespaceIDs error and I fixed by manually updating the namespace ID. I restarted the cluster and tried running sample Pi calculation job. It hangs as shown below

[cid:[EMAIL PROTECTED]1660]

It is running for more than 30 minutes. I quickly checked the logs and did not find any exception. How do I fix this? Any help would be greatly appreciated.

Regards,
Anand.C
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB