-Re: How to get hadoop job information effectively?
Robert Evans 2011-09-21, 18:47
Not that I know of. We scrape web pages which is a horrible thing to do. There is a JIRA to add in some web service APIs to expose this type of information, but it is not going to be available for a while.
On 9/21/11 1:01 PM, "Benyi Wang" <[EMAIL PROTECTED]> wrote:
I'm working a project to collect MapReduce job information on an application
level. For example, a DW ETL process may involves several MapReduce jobs, we
want to have a dashboard to show the progress of those jobs for the specific
JobStatus does not provide all information like JobTracker web
page. JobInProgress is used in JobTracker and JobHistory and it is in
JobTracker memory, and seem not exposed to the client side.
The current method I am using is to check history log files and job conf XML
file to extract those information like jobdetailhistory.jsp and
Is there a better way to collect the information like JobInProgress?