Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Number of mapreduce job and the time spent


Copy link to this message
-
Number of mapreduce job and the time spent
Hi,

 I want to know from the output of the execution of the example of
mapreduce  wordcount on hadoop : the number of mapreduce job and the time
spent for the execution.

There is an exceprt from the output.

12/12/12 10:20:09 INFO mapred.Task: Task 'attempt_local_0001_r_000000_0'
done.
12/12/12 10:20:10 INFO mapred.JobClient:  map 100% reduce 100%
12/12/12 10:20:10 INFO mapred.JobClient: Job complete: job_local_0001
12/12/12 10:20:10 INFO mapred.JobClient: Counters: 22
12/12/12 10:20:10 INFO mapred.JobClient:   File Input Format Counters
12/12/12 10:20:10 INFO mapred.JobClient:     Bytes Read=145966941
12/12/12 10:20:10 INFO mapred.JobClient:   File Output Format Counters
12/12/12 10:20:10 INFO mapred.JobClient:     Bytes Written=50704638
12/12/12 10:20:10 INFO mapred.JobClient:   org.myorg.WordCount$Map$Counters
12/12/12 10:20:10 INFO mapred.JobClient:     INPUT_WORDS=4980060
12/12/12 10:20:10 INFO mapred.JobClient:   FileSystemCounters
12/12/12 10:20:10 INFO mapred.JobClient:     FILE_BYTES_READ=1777104865
12/12/12 10:20:10 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=1783494521
12/12/12 10:20:10 INFO mapred.JobClient:   Map-Reduce Framework
12/12/12 10:20:10 INFO mapred.JobClient:     Map output materialized
bytes=170854986
12/12/12 10:20:10 INFO mapred.JobClient:     Map input records=4980060
12/12/12 10:20:10 INFO mapred.JobClient:     Reduce shuffle bytes=0
12/12/12 10:20:10 INFO mapred.JobClient:     Spilled Records=14940180
12/12/12 10:20:10 INFO mapred.JobClient:     Map output bytes=160894830
12/12/12 10:20:10 INFO mapred.JobClient:     Total committed heap usage
(bytes)=1185910784
12/12/12 10:20:10 INFO mapred.JobClient:     CPU time spent (ms)=0
12/12/12 10:20:10 INFO mapred.JobClient:     Map input bytes=145954650
12/12/12 10:20:10 INFO mapred.JobClient:     SPLIT_RAW_BYTES=614
12/12/12 10:20:10 INFO mapred.JobClient:     Combine input records=8426541
12/12/12 10:20:10 INFO mapred.JobClient:     Reduce input records=4980060
12/12/12 10:20:10 INFO mapred.JobClient:     Reduce input groups=1660020
12/12/12 10:20:10 INFO mapred.JobClient:     Combine output records=8426541
12/12/12 10:20:10 INFO mapred.JobClient:     Physical memory (bytes)
snapshot=0
12/12/12 10:20:10 INFO mapred.JobClient:     Reduce output records=1660020
12/12/12 10:20:10 INFO mapred.JobClient:     Virtual memory (bytes)
snapshot=0
12/12/12 10:20:10 INFO mapred.JobClient:     Map output records=4980060
Thank you for your responses.