Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> why i can not track the job which i submitted in yarn?


Copy link to this message
-
RE: why i can not track the job which i submitted in yarn?
Your Job is running in local mode, that's why you don't see in the RM UI/Job History.

Can you change 'mapreduce.framework.name' configuration value to 'yarn', it will show in RM UI.

Thanks
Devaraj k

From: ch huang [mailto:[EMAIL PROTECTED]]
Sent: 11 September 2013 15:08
To: [EMAIL PROTECTED]
Subject: why i can not track the job which i submitted in yarn?

hi,all:
     i do now know why i can not track my job which submitted to yarn ?

# hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.3.0.jar pi 20 10
Number of Maps  = 20
Samples per Map = 10
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Wrote input for Map #10
Wrote input for Map #11
Wrote input for Map #12
Wrote input for Map #13
Wrote input for Map #14
Wrote input for Map #15
Wrote input for Map #16
Wrote input for Map #17
Wrote input for Map #18
Wrote input for Map #19
Starting Job
13/09/11 17:32:02 WARN conf.Configuration: session.id<http://session.id> is deprecated. Instead, use dfs.metrics.session-id
13/09/11 17:32:02 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId13/09/11 17:32:02 WARN conf.Configuration: slave.host.name<http://slave.host.name> is deprecated. Instead, use dfs.datanode.hostname
13/09/11 17:32:02 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/09/11 17:32:02 INFO mapred.FileInputFormat: Total input paths to process : 20
13/09/11 17:32:03 INFO mapred.LocalJobRunner: OutputCommitter set in config null
13/09/11 17:32:03 INFO mapred.JobClient: Running job: job_local854997782_0001
13/09/11 17:32:03 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
13/09/11 17:32:03 INFO mapred.LocalJobRunner: Waiting for map tasks
13/09/11 17:32:03 INFO mapred.LocalJobRunner: Starting task: attempt_local854997782_0001_m_000000_0
13/09/11 17:32:03 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
13/09/11 17:32:03 INFO util.ProcessTree: setsid exited with exit code 0
13/09/11 17:32:03 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7f342545<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7f342545>
13/09/11 17:32:03 INFO mapred.MapTask: Processing split: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part0:0+118
13/09/11 17:32:03 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group name and  BYTES_READ as counter name instead
13/09/11 17:32:03 INFO mapred.MapTask: numReduceTasks: 1
13/09/11 17:32:03 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
13/09/11 17:32:03 INFO mapred.MapTask: io.sort.mb = 100
13/09/11 17:32:03 INFO mapred.MapTask: data buffer = 79691776/99614720
13/09/11 17:32:03 INFO mapred.MapTask: record buffer = 262144/327680
13/09/11 17:32:03 INFO mapred.MapTask: Starting flush of map output
13/09/11 17:32:03 INFO mapred.MapTask: Finished spill 0
13/09/11 17:32:03 INFO mapred.Task: Task:attempt_local854997782_0001_m_000000_0 is done. And is in the process of commiting
13/09/11 17:32:03 INFO mapred.LocalJobRunner: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part0:0+118
13/09/11 17:32:03 INFO mapred.Task: Task 'attempt_local854997782_0001_m_000000_0' done.
13/09/11 17:32:03 INFO mapred.LocalJobRunner: Finishing task: attempt_local854997782_0001_m_000000_0
13/09/11 17:32:03 INFO mapred.LocalJobRunner: Starting task: attempt_local854997782_0001_m_000001_0
13/09/11 17:32:03 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
13/09/11 17:32:03 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@18f63055<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@18f63055>
13/09/11 17:32:03 INFO mapred.MapTask: Processing split: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part1:0+118
13/09/11 17:32:04 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group name and  BYTES_READ as counter name instead
13/09/11 17:32:04 INFO mapred.MapTask: numReduceTasks: 1
13/09/11 17:32:04 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
13/09/11 17:32:04 INFO mapred.MapTask: io.sort.mb = 100
13/09/11 17:32:04 INFO mapred.MapTask: data buffer = 79691776/99614720
13/09/11 17:32:04 INFO mapred.MapTask: record buffer = 262144/327680
13/09/11 17:32:04 INFO mapred.MapTask: Starting flush of map output
13/09/11 17:32:04 INFO mapred.MapTask: Finished spill 0
13/09/11 17:32:04 INFO mapred.Task: Task:attempt_local854997782_0001_m_000001_0 is done. And is in the process of commiting
13/09/11 17:32:04 INFO mapred.LocalJobRunner: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part1:0+118
13/09/11 17:32:04 INFO mapred.Task: Task 'attempt_local854997782_0001_m_000001_0' done.
13/09/11 17:32:04 INFO mapred.LocalJobRunner: Finishing task: attempt_local854997782_0001_m_000001_0
13/09/11 17:32:04 INFO mapred.LocalJobRunner: Starting task: attempt_local854997782_0001_m_000002_0
13/09/11 17:32:04 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
13/09/11 17:32:04 INFO mapred.Task:  Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@74bfd10a<mailto:org.apache.hadoop.util.LinuxResourceCalculatorPlugin@74bfd10a>
13/09/11 17:32:04 INFO mapred.MapTask: Processing split: hdfs://CH22:9000/user/root/PiEstimator_TMP_3_141592654/in/part10:0+118
13/09/11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB