Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> MR job output report


Copy link to this message
-
MR job output report
All,Q. How does everyone refer to the typical output report of a MapReduce job (below)? What is it called?Q. Isn't that "report" saved somewhere? Is it reproduciable? I've looked at the HDFS job output directory which contains a log, but that doesn't have all the info as the report.Q. How can I redirect the output of the distcp (next line) to a logfile. I've tried alot of the     typical command line redirects(>,>>,&>,2>..). They didn't work.:hadoop distcp -i hdfs://had2100.zim.com:8020/akamai/2013/02/01                  hdfs://zip4.zim.com/akamai/2013/02/$n
many thanks,John-----Typical MR output report------13/03/11 13:35:53 INFO mapred.LocalJobRunner: Copied: 495 Skipped: 0 Failed: 013/03/11 13:35:53 INFO mapred.Task: Task 'attempt_local1065909426_0001_m_000000_0' done.13/03/11 13:35:53 INFO mapred.LocalJobRunner: Finishing task: attempt_local1065909426_0001_m_000000_013/03/11 13:35:53 INFO mapred.LocalJobRunner: Map task executor complete.13/03/11 13:35:53 INFO mapred.JobClient:  map 100% reduce 0%13/03/11 13:35:53 INFO mapred.JobClient: Job complete: job_local1065909426_000113/03/11 13:35:53 INFO mapred.JobClient: Counters: 2213/03/11 13:35:53 INFO mapred.JobClient:   File System Counters13/03/11 13:35:53 INFO mapred.JobClient:     FILE: Number of bytes read=52564013/03/11 13:35:53 INFO mapred.JobClient:     FILE: Number of bytes written=51899613/03/11 13:35:53 INFO mapred.JobClient:     FILE: Number of read operations=013/03/11 13:35:53 INFO mapred.JobClient:     FILE: Number of large read operations=013/03/11 13:35:53 INFO mapred.JobClient:     FILE: Number of write operations=013/03/11 13:35:53 INFO mapred.JobClient:     HDFS: Number of bytes read=2750597724213/03/11 13:35:53 INFO mapred.JobClient:     HDFS: Number of bytes written=2750597724213/03/11 13:35:53 INFO mapred.JobClient:     HDFS: Number of read operations=302613/03/11 13:35:53 INFO mapred.JobClient:     HDFS: Number of large read operations=013/03/11 13:35:53 INFO mapred.JobClient:     HDFS: Number of write operations=151413/03/11 13:35:53 INFO mapred.JobClient:   Map-Reduce Framework13/03/11 13:35:53 INFO mapred.JobClient:     Map input records=51913/03/11 13:35:53 INFO mapred.JobClient:     Map output records=013/03/11 13:35:53 INFO mapred.JobClient:     Input split bytes=14513/03/11 13:35:53 INFO mapred.JobClient:     Spilled Records=013/03/11 13:35:53 INFO mapred.JobClient:     CPU time spent (ms)=013/03/11 13:35:53 INFO mapred.JobClient:     Physical memory (bytes) snapshot=013/03/11 13:35:53 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=013/03/11 13:35:53 INFO mapred.JobClient:     Total committed heap usage (bytes)=40946892813/03/11 13:35:53 INFO mapred.JobClient:   org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter13/03/11 13:35:53 INFO mapred.JobClient:     BYTES_READ=10206613/03/11 13:35:53 INFO mapred.JobClient:   distcp13/03/11 13:35:53 INFO mapred.JobClient:     Bytes copied=2750597724213/03/11 13:35:53 INFO mapred.JobClient:     Bytes expected=2750597724213/03/11 13:35:53 INFO mapred.JobClient:     Files copied=495
     
+
Jagmohan Chauhan 2013-03-11, 22:36