Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Job History files in Hadoop 2.0


+
sangroya 2013-02-05, 15:42
Copy link to this message
-
Re: Job History files in Hadoop 2.0
Hi Amit,

The job history files are stored in HDFS. eg. /mapred/history/done/2013/02/06/000000

I would think that there would be some changes in the format.

Thanks
Ravi
________________________________
 From: sangroya <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, February 5, 2013 9:42 AM
Subject: Job History files in Hadoop 2.0
 
Hi,

I recently migrated to Hadoop 2.0 from Hadoop 1.0 (0.20.2 before).

I am able to successfully launch example applications.

/Could anyone please suggest where are the MapReduce job history files
available, after running jobs in Hadoop 2.0./

I need the statistics after running the jobs. Of course, the web UI gives me
the information. But I need the history files that were available in the
previous versions of hadoop job_ID etc.

I can see a directory with application and container patterns but this does
not have specific information about job submit time, Map Start time, finish
time, reduce start time, finish time, job finish time etc.

In the previous version of hadoop i.e. 1.0 or 0.20, it was stored under
logs/history.

Can anyone suggest if the pattern of storing job history files is also
changed in the new architecture?
Thanks in advance,
Amit

-----
Sangroya
--
View this message in context: http://lucene.472066.n3.nabble.com/Job-History-files-in-Hadoop-2-0-tp4038599.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.