Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - why i can not track the job which i submitted in yarn?


Copy link to this message
-
RE: why i can not track the job which i submitted in yarn?
Devaraj k 2013-09-11, 09:51
Your Job is running in local mode, that's why you don't see in the RM UI/Job History.

Can you change 'mapreduce.framework.name' configuration value to 'yarn', it will show in RM UI.

Thanks
Devaraj k

From: ch huang [mailto:[EMAIL PROTECTED]]
Sent: 11 September 2013 15:08
To: [EMAIL PROTECTED]
Subject: why i can not track the job which i submitted in yarn?

hi,all:
     i do now know why i can not track my job which submitted to yarn ?

# hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar pi 20 10
Number of Maps  = 20
Samples per Map = 10
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Wrote input for Map #10
Wrote input for Map #11
Wrote input for Map #12
Wrote input for Map #13
Wrote input for Map #14
Wrote input for Map #15
Wrote input for Map #16
Wrote input for Map #17
Wrote input for Map #18
Wrote input for Map #19
Starting Job
13/09/11 17:32:02 WARN conf.Configuration: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
13/09/11 17:32:02 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId13/09/11 17:32:02 WARN conf.Configuration: slave.host.name<http://slave.host.name> is deprecated. Instead, use dfs.datanode.hostname
13/09/11 17:32:02 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/09/11 17:32:02 INFO mapred.FileInputFormat: Total input paths to process : 20
13/09/11 17:32:03 INFO mapred.LocalJobRunner: OutputCommitter set in config null
13/09/11 17:32:03 INFO mapred.JobClient: Running job: job_local854997782_0001
13/09/11 17:32:03 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
13/09/11 17:32:03 INFO mapred.LocalJobRunner: Waiting for map tasks
13/09/11 17:32:03 INFO mapred.LocalJobRunner: Starting task: attempt_local854997782_0001_m_000000_0
13/09/11 17:32:03 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
13/09/11 17:32:03 INFO util.ProcessTree: setsid exited with exit code 0
13/09/11 17:32:03 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7f342545<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7f342545>
13/09/11 17:32:03 INFO mapred.MapTask: Processing split: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part0:0+118
13/09/11 17:32:03 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group name and  BYTES_READ as counter name instead
13/09/11 17:32:03 INFO mapred.MapTask: numReduceTasks: 1
13/09/11 17:32:03 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
13/09/11 17:32:03 INFO mapred.MapTask: io.sort.mb = 100
13/09/11 17:32:03 INFO mapred.MapTask: data buffer = 79691776/99614720
13/09/11 17:32:03 INFO mapred.MapTask: record buffer = 262144/327680
13/09/11 17:32:03 INFO mapred.MapTask: Starting flush of map output
13/09/11 17:32:03 INFO mapred.MapTask: Finished spill 0
13/09/11 17:32:03 INFO mapred.Task: Task:attempt_local854997782_0001_m_000000_0 is done. And is in the process of commiting
13/09/11 17:32:03 INFO mapred.LocalJobRunner: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part0:0+118
13/09/11 17:32:03 INFO mapred.Task: Task 'attempt_local854997782_0001_m_000000_0' done.
13/09/11 17:32:03 INFO mapred.LocalJobRunner: Finishing task: attempt_local854997782_0001_m_000000_0
13/09/11 17:32:03 INFO mapred.LocalJobRunner: Starting task: attempt_local854997782_0001_m_000001_0
13/09/11 17:32:03 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
13/09/11 17:32:03 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@18f63055<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@18f63055>
13/09/11 17:32:03 INFO mapred.MapTask: Processing split: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part1:0+118
13/09/11 17:32:04 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group name and  BYTES_READ as counter name instead
13/09/11 17:32:04 INFO mapred.MapTask: numReduceTasks: 1
13/09/11 17:32:04 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
13/09/11 17:32:04 INFO mapred.MapTask: io.sort.mb = 100
13/09/11 17:32:04 INFO mapred.MapTask: data buffer = 79691776/99614720
13/09/11 17:32:04 INFO mapred.MapTask: record buffer = 262144/327680
13/09/11 17:32:04 INFO mapred.MapTask: Starting flush of map output
13/09/11 17:32:04 INFO mapred.MapTask: Finished spill 0
13/09/11 17:32:04 INFO mapred.Task: Task:attempt_local854997782_0001_m_000001_0 is done. And is in the process of commiting
13/09/11 17:32:04 INFO mapred.LocalJobRunner: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part1:0+118
13/09/11 17:32:04 INFO mapred.Task: Task 'attempt_local854997782_0001_m_000001_0' done.
13/09/11 17:32:04 INFO mapred.LocalJobRunner: Finishing task: attempt_local854997782_0001_m_000001_0
13/09/11 17:32:04 INFO mapred.LocalJobRunner: Starting task: attempt_local854997782_0001_m_000002_0
13/09/11 17:32:04 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
13/09/11 17:32:04 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@74bfd10a<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@74bfd10a>
13/09/11 17:32:04 INFO mapred.MapTask: Processing split: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part10:0+118
13/09/11