Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> RE: How to configure container capacity?


Copy link to this message
-
RE: How to configure container capacity?
Bcc'd dev mailing list.

Hi Andrew,

The memory allocated will always be an integral multiple of minimal allocation unit which is configured via the property "yarn.scheduler.minimum-allocation-mb ". The default configuration for the value is 1024.
If you change the config to 512, the container memory size should be 512 with -Dmapreduce.map.memory.mb=512.

Thanks,
Chuan

-----Original Message-----
From: Yuzhang Han [mailto:[EMAIL PROTECTED]]
Sent: Friday, May 31, 2013 12:24 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: How to configure container capacity?

Hi,

Can anyone tell me how to configure the capacity of containers? Which parameters should be set?

I use the following command with -Dmapreduce.map.memory.mb=512, but why is the map container size always 1024?

$HADOOP_COMMON_HOME/bin/hadoop jar
$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.4-alpha.jar
randomwriter -Dmapreduce.job.user.name=$USER
-Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.block.size=536870912
-Dmapreduce.map.memory.mb=512 output

Thank you,
Andrew
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB