I am building a monitoring tool for my Hadoop cluster. I have been able to
collect most of the data I need from JobClient.
How can I get JobConf for a job? My monitoring tool needs the outputPath and
inputPath of the mapreduce jobs.

I appreciate any hint you can give me.

Thanks,
Patek

Hi All,
I am building a monitoring tool for my Hadoop cluster. I have been able to
collect most of the data I need from JobClient.
How can I get JobConf for a job? My monitoring tool needs the outputPath and
inputPath of the mapreduce jobs.

I appreciate any hint you can give me.

Thanks,
Patek

Check $HADOOP_HOME/logs directory where you can find:
job_201008102250_0035_conf.xml  job_201008102250_0092_conf.xml  ...

On Thu, Aug 12, 2010 at 10:58 AM, patek tek <[EMAIL PROTECTED]> wrote:

> Hi All,
> I am building a monitoring tool for my Hadoop cluster. I have been able to
> collect most of the data I need from JobClient.
> How can I get JobConf for a job? My monitoring tool needs the outputPath
> and
> inputPath of the mapreduce jobs.
>
> I appreciate any hint you can give me.
>
> Thanks,
> Patek
>

Thanks for your response, Ted.
Yes, that I did but requires the tool to gain access to the log files and
parse the xml file etc,
I was hoping that i could get that data from  the jobtracker

Any other way?
Patek

On Thu, Aug 12, 2010 at 2:35 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Check $HADOOP_HOME/logs directory where you can find:
> job_201008102250_0035_conf.xml  job_201008102250_0092_conf.xml  ...
>
> On Thu, Aug 12, 2010 at 10:58 AM, patek tek <[EMAIL PROTECTED]> wrote:
>
> > Hi All,
> > I am building a monitoring tool for my Hadoop cluster. I have been able
> to
> > collect most of the data I need from JobClient.
> > How can I get JobConf for a job? My monitoring tool needs the outputPath
> > and
> > inputPath of the mapreduce jobs.
> >
> > I appreciate any hint you can give me.
> >
> > Thanks,
> > Patek
> >
>

The JT saves job-history and job.xml for each job in HDFS to a path  
specified by "mapred.job.tracker.history.completed.location"

Hope that helps.

Arun

On Aug 11, 2010, at 8:21 PM, patek tek wrote:

> I am building a monitoring tool for my Hadoop cluster. I have been  
> able to
> collect most of the data I need from JobClient.
> How can I get JobConf for a job? My monitoring tool needs the  
> outputPath and
> inputPath of the mapreduce jobs.
>
> I appreciate any hint you can give me.
>
> Thanks,
> Patek