Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> How many machines did my MR job used?


+
Something Something 2013-06-21, 00:21
Copy link to this message
-
Re: How many machines did my MR job used?
When Pig launches each job, it prints a JobTracker URL on console. For
example,

More information at:
http://<host>:<port>/jobdetails.jsp?jobid=job_X<http://10.11.46.215:9100/jobdetails.jsp?jobid=job_201306032155_59754>

Now you can find the jobtracker log and grep the job id. Then, you will see
all the task attempts associated with that job as well as machines where
the tasks are running. For example,

'attempt_X_m_000888_0' to tip task_X_m_000888, for tracker
'tracker_<host>:localhost/127.0.0.1: <http://127.0.0.1:33768><port>'

Alternatively, on the JobTracker UI, you can follow the link and will find
the job page. Click on the number of running tasks will take you to tasks
page. Lastly, click on each task link will show you which machine it is
running on.
On Thu, Jun 20, 2013 at 5:21 PM, Something Something <
[EMAIL PROTECTED]> wrote:

> Hello,
>
> I am running a Pig script which internally starts several jobs.  For one of
> the jobs that uses maximum no. of mappers & reducers, I need to find out
> how many machines it's running on & which machines are those.
>
> I looked around the JobTracker UI, but couldn't find this information.  Is
> it there somewhere?  Is it in any of the logs?
>
> Would appreciate any help on this.  Thanks.
>
+
Arun C Murthy 2013-06-21, 13:57