Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: reg max map task config


Copy link to this message
-
Re: reg max map task config
There's probably different variables (esp. since you run HBase,
calculated resources) involved for this, but you can generally find
some reading material on this in books and
http://wiki.apache.org/hadoop/HowManyMapsAndReduces which should help
you come to a good value. Pro-tip I usually get from Todd:
Consistently monitor and tune, for it is harmless to restart a slave
to set in new values.

On Mon, Jan 28, 2013 at 6:04 PM, Jean-Marc Spaggiari
<[EMAIL PROTECTED]> wrote:
> Hi Harsh,
>
> Is there a recommanded way to configure this value? I think I read
> something like 0.7 x cores but I'm not 100% sure...
>
> JM
>
> 2013/1/28, Manoj Babu <[EMAIL PROTECTED]>:
>> Thank you Harsh.
>>
>> Cheers!
>> Manoj.
>>
>>
>> On Mon, Jan 28, 2013 at 10:50 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> As noted on
>>> http://hadoop.apache.org/docs/stable/cluster_setup.html#Configuring+the+Hadoop+Daemons
>>> ,
>>> and by its own name, the config 'mapred.tasktracker.map.tasks.maximum'
>>> applies to a TaskTracker daemon and therefore needs to be applied to
>>> its local mapred-site.xml and has to be restarted to take new values
>>> into effect.
>>>
>>> On Mon, Jan 28, 2013 at 10:36 AM, Manoj Babu <[EMAIL PROTECTED]> wrote:
>>> > Hi All,
>>> >
>>> > I am trying to override the value of
>>> mapred.tasktracker.map.tasks.maximum in
>>> > mapred-site.xml through my job configuration object will it reflect in
>>> the
>>> > corresponding task trackers?
>>> > or do i have to change it in mapred-site.xml following a restart for
>>> > the
>>> > changes to apply?
>>> >
>>> > Thanks in advance.
>>> >
>>> > Cheers!
>>> > Manoj.
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB