Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> mapred.tasktacker.reduce.tasks.maximum issue


Copy link to this message
-
Re: mapred.tasktacker.reduce.tasks.maximum issue
Dear Vincent,

You have to make sure that mapred-site is modified on every tasktracker
node! Not just the master! Mapred-site can be customized if you have a
heterogeneous cluster. I usually start the dfs, then copy all the
mapredsites to the slave nodes, then run start-mapred.sh and then start a
job.

Regards, Dieter
2013/11/14 Vincent Y. Shen <[EMAIL PROTECTED]>

> Hi Guys,
>
> I am increasing mapred.tasktacker.reduce.tasks.maximum in the hadoop
> cluster. But regardless whatever I tried, hadoop refuses to use my value
> but the default (2)
>
>
>
>
> What I did do
> ==========>
> stop mapred
> stop dfs
>
> update the value of mapred.tasktacker.reduce.tasks.maximum
>
>
> start dfs
> start mapred
>
>
> But the system still uses default values
>
>
>
>
> Environment
> ========>
> Hadoop version: 1.1.2
> Java version: 7
>
>
> Mapred-site.xml
> ===========>
> <property>
>   <name>mapred.system.dir</name>
>   <value>/hadoop/mapred/system</value>
> </property>
>
>
>
> <property>
>   <name>mapred.reduce.tasks</name>
>   <value>50</value>
> </property>
>
> <property>
>   <name>mapred.tasktacker.reduce.tasks.maximum</name>
>   <value>15</value>
> </property>
>
> <property>
>   <name>mapred.tasktacker.map.tasks.maximum</name>
>   <value>15</value>
> </property>
>
>
> <property>
>   <name>mapred.child.java.opts</name>
>   <value>-Xmx500m</value>
> </property>
>
>
>
> <property>
>   <name>mapred.job.tracker</name>
>   <value>machine1.mydomain.com:54311</value>
>
> </property>
>
>
>
> Thanks guys for any pointers!!!!
>
>
> Best regards,
>
>
> Vincent.
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB