Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: reg max map task config


Copy link to this message
-
Re: reg max map task config
Harsh J 2013-01-28, 16:23
There's probably different variables (esp. since you run HBase,
calculated resources) involved for this, but you can generally find
some reading material on this in books and
http://wiki.apache.org/hadoop/HowManyMapsAndReduces which should help
you come to a good value. Pro-tip I usually get from Todd:
Consistently monitor and tune, for it is harmless to restart a slave
to set in new values.

On Mon, Jan 28, 2013 at 6:04 PM, Jean-Marc Spaggiari
<[EMAIL PROTECTED]> wrote:
> Hi Harsh,
>
> Is there a recommanded way to configure this value? I think I read
> something like 0.7 x cores but I'm not 100% sure...
>
> JM
>
> 2013/1/28, Manoj Babu <[EMAIL PROTECTED]>:
>> Thank you Harsh.
>>
>> Cheers!
>> Manoj.
>>
>>
>> On Mon, Jan 28, 2013 at 10:50 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> As noted on
>>> http://hadoop.apache.org/docs/stable/cluster_setup.html#Configuring+the+Hadoop+Daemons
>>> ,
>>> and by its own name, the config 'mapred.tasktracker.map.tasks.maximum'
>>> applies to a TaskTracker daemon and therefore needs to be applied to
>>> its local mapred-site.xml and has to be restarted to take new values
>>> into effect.
>>>
>>> On Mon, Jan 28, 2013 at 10:36 AM, Manoj Babu <[EMAIL PROTECTED]> wrote:
>>> > Hi All,
>>> >
>>> > I am trying to override the value of
>>> mapred.tasktracker.map.tasks.maximum in
>>> > mapred-site.xml through my job configuration object will it reflect in
>>> the
>>> > corresponding task trackers?
>>> > or do i have to change it in mapred-site.xml following a restart for
>>> > the
>>> > changes to apply?
>>> >
>>> > Thanks in advance.
>>> >
>>> > Cheers!
>>> > Manoj.
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>

--
Harsh J