Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> mapred.tasktacker.reduce.tasks.maximum issue


+
Vincent Y. Shen 2013-11-14, 14:53
+
Dieter De Witte 2013-11-14, 14:56
+
Vincent Y. Shen 2013-11-14, 14:58
Copy link to this message
-
Re: mapred.tasktacker.reduce.tasks.maximum issue
you could make the values final: add <final> true </final> to make sure
they cannot be overridden..
2013/11/14 Vincent Y. Shen <[EMAIL PROTECTED]>

> Hi Dieter,
>
> Thanks a lot for prompt reply! very much appreciated.
>
> Indeed, I did change in the mapred-site.xml in ALL the nodes and restarted
> the cluster 4 times but no luck...
>
>
> Any other hint?
>
>
>
>
>
>
> On Thu, Nov 14, 2013 at 2:56 PM, Dieter De Witte <[EMAIL PROTECTED]>wrote:
>
>> Dear Vincent,
>>
>> You have to make sure that mapred-site is modified on every tasktracker
>> node! Not just the master! Mapred-site can be customized if you have a
>> heterogeneous cluster. I usually start the dfs, then copy all the
>> mapredsites to the slave nodes, then run start-mapred.sh and then start a
>> job.
>>
>> Regards, Dieter
>>
>>
>> 2013/11/14 Vincent Y. Shen <[EMAIL PROTECTED]>
>>
>> Hi Guys,
>>>
>>> I am increasing mapred.tasktacker.reduce.tasks.maximum in the hadoop
>>> cluster. But regardless whatever I tried, hadoop refuses to use my value
>>> but the default (2)
>>>
>>>
>>>
>>>
>>> What I did do
>>> ==========>>>
>>> stop mapred
>>> stop dfs
>>>
>>> update the value of mapred.tasktacker.reduce.tasks.maximum
>>>
>>>
>>> start dfs
>>> start mapred
>>>
>>>
>>> But the system still uses default values
>>>
>>>
>>>
>>>
>>> Environment
>>> ========>>>
>>> Hadoop version: 1.1.2
>>> Java version: 7
>>>
>>>
>>> Mapred-site.xml
>>> ===========>>>
>>> <property>
>>>   <name>mapred.system.dir</name>
>>>   <value>/hadoop/mapred/system</value>
>>> </property>
>>>
>>>
>>>
>>> <property>
>>>   <name>mapred.reduce.tasks</name>
>>>   <value>50</value>
>>> </property>
>>>
>>> <property>
>>>   <name>mapred.tasktacker.reduce.tasks.maximum</name>
>>>   <value>15</value>
>>> </property>
>>>
>>> <property>
>>>   <name>mapred.tasktacker.map.tasks.maximum</name>
>>>   <value>15</value>
>>> </property>
>>>
>>>
>>> <property>
>>>   <name>mapred.child.java.opts</name>
>>>   <value>-Xmx500m</value>
>>> </property>
>>>
>>>
>>>
>>> <property>
>>>   <name>mapred.job.tracker</name>
>>>   <value>machine1.mydomain.com:54311</value>
>>>
>>> </property>
>>>
>>>
>>>
>>> Thanks guys for any pointers!!!!
>>>
>>>
>>> Best regards,
>>>
>>>
>>> Vincent.
>>>
>>>
>>>
>>
>
+
Vincent Y. Shen 2013-11-14, 15:53
+
Zizon Qiu 2013-11-15, 01:57
+
Adam Kawa 2013-11-27, 23:43
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB