Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> RE: How to configure container capacity?


Copy link to this message
-
RE: How to configure container capacity?
Bcc'd dev mailing list.

Hi Andrew,

The memory allocated will always be an integral multiple of minimal allocation unit which is configured via the property "yarn.scheduler.minimum-allocation-mb ". The default configuration for the value is 1024.
If you change the config to 512, the container memory size should be 512 with -Dmapreduce.map.memory.mb=512.

Thanks,
Chuan

-----Original Message-----
From: Yuzhang Han [mailto:[EMAIL PROTECTED]]
Sent: Friday, May 31, 2013 12:24 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: How to configure container capacity?

Hi,

Can anyone tell me how to configure the capacity of containers? Which parameters should be set?

I use the following command with -Dmapreduce.map.memory.mb=512, but why is the map container size always 1024?

$HADOOP_COMMON_HOME/bin/hadoop jar
$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.4-alpha.jar
randomwriter -Dmapreduce.job.user.name=$USER
-Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.block.size=536870912
-Dmapreduce.map.memory.mb=512 output

Thank you,
Andrew