Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Hadoop 2.2.0 MR tasks failing


Copy link to this message
-
Hadoop 2.2.0 MR tasks failing
Robert Dyer 2013-10-22, 04:55
I recently setup a 2.2.0 test cluster.  For some reason, all of my MR jobs
are failing.  The maps and reduces all run to completion, without any
errors.  Yet the app is marked failed and there is no final output.  Any
ideas?

Application Type: MAPREDUCE
State: FINISHED
FinalStatus: FAILED
Diagnostics: We crashed durring a commit

I notice in the logs this (but not sure what to make of it):

2013-10-21 23:42:41,379 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 789 for container-id
container_1382415258498_0002_01_000001: 250.4 MB of 2 GB physical
memory used; 2.0 GB of 6 GB virtual memory used
2013-10-21 23:42:41,743 WARN
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
Exit code from container container_1382415258498_0002_01_000001 is :
255
2013-10-21 23:42:41,744 WARN
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
Exception from container-launch with container ID:
container_1382415258498_0002_01_000001 and exit code: 255
org.apache.hadoop.util.Shell$ExitCodeException:

2013-10-21 23:42:41,746 INFO
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
2013-10-21 23:42:41,747 WARN
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch:
Container exited with a non-zero exit code 255
2013-10-21 23:42:41,747 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_1382415258498_0002_01_000001 transitioned from
RUNNING to EXITED_WITH_FAILURE
2013-10-21 23:42:41,747 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch:
Cleaning up container container_1382415258498_0002_01_000001
2013-10-21 23:42:41,764 INFO
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
Deleting absolute path :
/hadoop/hadoop-2.2.0/cluster-data/usercache/hadoop/appcache/application_1382415258498_0002/container_1382415258498_0002_01_000001
2013-10-21 23:42:41,765 WARN
org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger:
USER=hadoop OPERATION=Container Finished -
Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container
failed with state:
EXITED_WITH_FAILURE APPID=application_1382415258498_0002 CONTAINERID=container_1382415258498_0002_01_000001