Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: In YARN, how does a task tracker knows the address of a job tracker?


Copy link to this message
-
Re: In YARN, how does a task tracker knows the address of a job tracker?
Omkar Joshi 2013-11-21, 19:06
Hi,

Starting with YARN there is no notion of job tracker and task tracker. Here
is a quick summary
JobTracker :-
1) Resource management :- Now done by Resource Manager (it does all
scheduling work)
2) Application state management :- managing and launching new map /reduce
tasks (done by Application Master .. It is per job not one single entity in
the cluster for all jobs like MRv1).
TaskTracker :- replaced by Node Manager

I would suggest you read the YARN blog
post<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>.
This will answer most of your questions. Plus read
this<http://www.slideshare.net/ovjforu/yarn-way-to-share-cluster-beyond>
(slide
12) for how job actually gets executed.

Thanks,
Omkar Joshi
*Hortonworks Inc.* <http://www.hortonworks.com>
On Thu, Nov 21, 2013 at 7:52 AM, ricky l <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I have a question of how a task tracker identifies job tracker address
> when I submit MR job through YARN. As far as I know, both job tracker and
> task trackers are launched through application master and I am curious
> about the details about job and task tracker launch sequence.
>
> thanks.
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.