Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - RE: How to configure container capacity?


Copy link to this message
-
RE: How to configure container capacity?
Chuan Liu 2013-05-31, 23:59
Bcc'd dev mailing list.

Hi Andrew,

The memory allocated will always be an integral multiple of minimal allocation unit which is configured via the property "yarn.scheduler.minimum-allocation-mb ". The default configuration for the value is 1024.
If you change the config to 512, the container memory size should be 512 with -Dmapreduce.map.memory.mb=512.

Thanks,
Chuan

-----Original Message-----
From: Yuzhang Han [mailto:[EMAIL PROTECTED]]
Sent: Friday, May 31, 2013 12:24 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: How to configure container capacity?

Hi,

Can anyone tell me how to configure the capacity of containers? Which parameters should be set?

I use the following command with -Dmapreduce.map.memory.mb=512, but why is the map container size always 1024?

$HADOOP_COMMON_HOME/bin/hadoop jar
$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.4-alpha.jar
randomwriter -Dmapreduce.job.user.name=$USER
-Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.block.size=536870912
-Dmapreduce.map.memory.mb=512 output

Thank you,
Andrew