Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Job stuck in running state on Hadoop 2.2.0


Copy link to this message
-
Re: Job stuck in running state on Hadoop 2.2.0
Adam Kawa 2013-12-10, 23:29
It sounds like the job was successfully submitted to the cluster, but there
as some problem when starting/running AM, so that no progress is made. It
happened to me once, when I was playing with YARN on a cluster consisting
of very small machines, and I mis-configured YARN to allocated to AM more
memory than the actual memory available on any machine on my cluster. So
that RM was not able to start AM anywhere due to inability to find big
enough container.

Could you show the logs from the job? The link should be available on your
console after you submit a job e.g.
13/12/10 10:41:21 INFO mapreduce.Job: The url to track the job:
http://compute-7-2:8088/proxy/application_1386668372725_0001/
2013/12/10 Silvina Caíno Lores <[EMAIL PROTECTED]>

> Thank you! I realized that, despite I exported the variables in the
> scripts, there were a few errors and my desired configuration wasn't being
> used (which explained other strange behavior).
>
> However, I'm still getting the same issue with the examples, for instance:
>
> hadoop jar
> ~/hadoop-2.2.0-maven/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar
> pi 1 100
> Number of Maps = 1
> Samples per Map = 100
> 13/12/10 10:41:18 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Wrote input for Map #0
> Starting Job
> 13/12/10 10:41:19 INFO client.RMProxy: Connecting to ResourceManager at /
> 0.0.0.0:8032
> 13/12/10 10:41:20 INFO input.FileInputFormat: Total input paths to process
> : 1
> 13/12/10 10:41:20 INFO mapreduce.JobSubmitter: number of splits:1
> 13/12/10 10:41:20 INFO Configuration.deprecation: user.name is
> deprecated. Instead, use mapreduce.job.user.name
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapred.jar is
> deprecated. Instead, use mapreduce.job.jar
> 13/12/10 10:41:20 INFO Configuration.deprecation:
> mapred.map.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.map.speculative
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapred.reduce.tasks is
> deprecated. Instead, use mapreduce.job.reduces
> 13/12/10 10:41:20 INFO Configuration.deprecation:
> mapred.output.value.class is deprecated. Instead, use
> mapreduce.job.output.value.class
> 13/12/10 10:41:20 INFO Configuration.deprecation:
> mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
> mapreduce.reduce.speculative
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapreduce.map.class is
> deprecated. Instead, use mapreduce.job.map.class
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapred.job.name is
> deprecated. Instead, use mapreduce.job.name
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapreduce.reduce.class
> is deprecated. Instead, use mapreduce.job.reduce.class
> 13/12/10 10:41:20 INFO Configuration.deprecation:
> mapreduce.inputformat.class is deprecated. Instead, use
> mapreduce.job.inputformat.class
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapred.output.dir is
> deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 13/12/10 10:41:20 INFO Configuration.deprecation:
> mapreduce.outputformat.class is deprecated. Instead, use
> mapreduce.job.outputformat.class
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapred.map.tasks is
> deprecated. Instead, use mapreduce.job.maps
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapred.output.key.class
> is deprecated. Instead, use mapreduce.job.output.key.class
> 13/12/10 10:41:20 INFO Configuration.deprecation: mapred.working.dir is
> deprecated. Instead, use mapreduce.job.working.dir
> 13/12/10 10:41:20 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1386668372725_0001
> 13/12/10 10:41:20 INFO impl.YarnClientImpl: Submitted application
> application_1386668372725_0001 to ResourceManager at /0.0.0.0:8032