|
|
+
Jean-Marc Spaggiari 2013-01-28, 12:34
-
Re: reg max map task configHarsh J 2013-01-28, 16:23
There's probably different variables (esp. since you run HBase,
calculated resources) involved for this, but you can generally find some reading material on this in books and http://wiki.apache.org/hadoop/HowManyMapsAndReduces which should help you come to a good value. Pro-tip I usually get from Todd: Consistently monitor and tune, for it is harmless to restart a slave to set in new values. On Mon, Jan 28, 2013 at 6:04 PM, Jean-Marc Spaggiari <[EMAIL PROTECTED]> wrote: > Hi Harsh, > > Is there a recommanded way to configure this value? I think I read > something like 0.7 x cores but I'm not 100% sure... > > JM > > 2013/1/28, Manoj Babu <[EMAIL PROTECTED]>: >> Thank you Harsh. >> >> Cheers! >> Manoj. >> >> >> On Mon, Jan 28, 2013 at 10:50 AM, Harsh J <[EMAIL PROTECTED]> wrote: >> >>> Hi, >>> >>> As noted on >>> http://hadoop.apache.org/docs/stable/cluster_setup.html#Configuring+the+Hadoop+Daemons >>> , >>> and by its own name, the config 'mapred.tasktracker.map.tasks.maximum' >>> applies to a TaskTracker daemon and therefore needs to be applied to >>> its local mapred-site.xml and has to be restarted to take new values >>> into effect. >>> >>> On Mon, Jan 28, 2013 at 10:36 AM, Manoj Babu <[EMAIL PROTECTED]> wrote: >>> > Hi All, >>> > >>> > I am trying to override the value of >>> mapred.tasktracker.map.tasks.maximum in >>> > mapred-site.xml through my job configuration object will it reflect in >>> the >>> > corresponding task trackers? >>> > or do i have to change it in mapred-site.xml following a restart for >>> > the >>> > changes to apply? >>> > >>> > Thanks in advance. >>> > >>> > Cheers! >>> > Manoj. >>> >>> >>> >>> -- >>> Harsh J >>> >> -- Harsh J |