Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - mapred.tasktracker.map.tasks.maximum


Copy link to this message
-
Re: mapred.tasktracker.map.tasks.maximum
Serge Blazhiyevskyy 2012-11-13, 15:27
;)

From: Mark Kerzner <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tuesday, November 13, 2012 5:25 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: mapred.tasktracker.map.tasks.maximum

Exactly! I found the right one, and it is 80.

Thank you,
Mark

On Tue, Nov 13, 2012 at 10:23 AM, Serge Blazhiyevskyy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Look on the job tracker web UI is parameter has taken effect (port 50030)

It might be in the wrong config file or something
Serge

From: Mark Kerzner <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>>
Date: Tuesday, November 13, 2012 5:19 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>>
Subject: Re: mapred.tasktracker.map.tasks.maximum

1.0.1

On Tue, Nov 13, 2012 at 10:18 AM, Serge Blazhiyevskyy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>> wrote:
What hadoop version are we talking about?

From: Mark Kerzner <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>>>
Date: Tuesday, November 13, 2012 5:16 PM
To: Hadoop User <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]><mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>>>
Subject: mapred.tasktracker.map.tasks.maximum

Hi,

I have a cluster with 4 nodes and 32 many cores on each. My default value for the maximum number of mappers per slot is 1:

  <property>
    <name>mapred.tasktracker.map.tasks.maximum</name>
    <!-- see other kb entry about this one. -->
    <value>1</value>
    <final>true</final>
  </property>
(which I think is wrong).

Howeve, when sizable jobs run, I see 65 mappers working, so it seems that it does create more than 1 mapper per node.

Questions: what maximum number of mappers would be appropriate in this situation? Is that the right way to set them?

Thank you. Sincerely,
Mark