Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - What are uses of taskTracker and JobTracker services?


Copy link to this message
-
Re: What are uses of taskTracker and JobTracker services?
Hemanth Yamijala 2010-06-30, 04:14
Hi,

> I think that he was trying to explain that in HDFS, you have a name node and then your data nodes.
> So you have the name node service on the name node and each data node has a data node service.
> When you run a map reduce job, you have a Job tracker that resides on the name node and controls the overall job.

May or may not be true. In general, for moderately complex cases, it
is best to run the name node and jobtracker on different nodes so both
masters don't fail where only one of them can.

> On each data node, where the jobs run in parallel, there exists a task tracker.

This is almost always true, of course - it helps Hadoop to achieve
data locality by colocating where the task runs with where it has to
read data from.

Thanks
Hemanth

>
>
>
>> Date: Tue, 29 Jun 2010 23:24:58 +0430
>> Subject: Re: What are uses of taskTracker and JobTracker services?
>> From: [EMAIL PROTECTED]
>> To: [EMAIL PROTECTED]
>>
>> Thanks for reply.
>>
>> But we have different services for each of them, for example a service for
>> NameNode and a Service for JobTracker, are they doing separate things each
>> one?
>>
>>
>>
>> On Tue, Jun 29, 2010 at 10:20 PM, Allen Wittenauer <[EMAIL PROTECTED]
>> > wrote:
>>
>> >
>> > On Jun 29, 2010, at 8:07 AM, Sarah kho wrote:
>> >
>> > > Hi,
>> > >
>> > > Can you please let me know what are tasks that the taskTracker and
>> > > JobTracker performs?
>> >
>> > Pretty much the entirety of the MapReduce framework.  You can think of it
>> > this way:
>> >
>> >
>> > HDFS <--> MR
>> > NameNode <--> JobTracker
>> > DataNode <--> TaskTracker
>
> _________________________________________________________________
> The New Busy is not the old busy. Search, chat and e-mail from your inbox.
> http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3