Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Hadoop 2.2.0 :  Job not showing on resource management web UI


Copy link to this message
-
Hadoop 2.2.0 :  Job not showing on resource management web UI
Hi,
I have set up a single node Hadoop 2.2.0 .
I can submit  wordcount  hadoop job to YARN from the command line.
It run successfully and I checked the result is in hdfs.
matmsh@gauss:~> hadoop jar  ~/installed/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar  wordcount input output6
14/06/28 13:40:00 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/06/28 13:40:00 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/06/28 13:40:00 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
14/06/28 13:40:00 INFO input.FileInputFormat: Total input paths to process : 1
14/06/28 13:40:00 INFO mapreduce.JobSubmitter: number of splits:1
14/06/28 13:40:00 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
14/06/28 13:40:00 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
14/06/28 13:40:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local28288787_0001
14/06/28 13:40:00 WARN conf.Configuration: file:/tmp/hadoop-matmsh/mapred/staging/matmsh28288787/.staging/job_local28288787_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
14/06/28 13:40:00 WARN conf.Configuration: file:/tmp/hadoop-matmsh/mapred/staging/matmsh28288787/.staging/job_local28288787_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
14/06/28 13:40:00 WARN conf.Configuration: file:/tmp/hadoop-matmsh/mapred/local/localRunner/matmsh/job_local28288787_0001/job_local28288787_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
14/06/28 13:40:00 WARN conf.Configuration: file:/tmp/hadoop-matmsh/mapred/local/localRunner/matmsh/job_local28288787_0001/job_local28288787_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
14/06/28 13:40:00 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
14/06/28 13:40:00 INFO mapreduce.Job: Running job: job_local28288787_0001
14/06/28 13:40:00 INFO mapred.LocalJobRunner: OutputCommitter set in config null
14/06/28 13:40:00 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
14/06/28 13:40:01 INFO mapred.LocalJobRunner: Waiting for map tasks
14/06/28 13:40:01 INFO mapred.LocalJobRunner: Starting task: attempt_local28288787_0001_m_000000_0
14/06/28 13:40:01 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
14/06/28 13:40:01 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/matmsh/input/getdoc.txt:0+23992
14/06/28 13:40:01 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
14/06/28 13:40:01 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
.....

However, the above job does not show up at  the web UI :
http://localhost:8088/cluster/apps

My configurations are below,
mapred-site.xml :
<configuration>

<property>
<value>mapreduce.framework.name</value>
<name>yarn</name>
</property>

</configuration>

hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>

<property>
<name>dfs.namenode.name.dir</name>
<value>file:/home/matmsh/temp/hadoop2/namenode</value>
<description>The name of the default file system.</description>
</property>

<property>
<name>dfs.datanode.data.dir</name>
<value>file:/home/matmsh/temp/hadoop2/datanode</value>
<description>The name of the default file system.</description>
</property>

</configuration>
yarn-site.xml
<configuration>

<!-- Site specific YARN configuration properties -->

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>
What other configurations are  needed to show the job in the web UI ?
Thanks in advance for any  assistance !

Shing