Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Cluster Tuning

Copy link to this message
Re: Cluster Tuning

On Jul 11, 2011, at 9:28 AM, Juan P. wrote:
> *  <property>*
> *    <name>mapred.child.java.opts</name>*
> *    <value>-Xmx400m</value>*
> *  </property>*

"Single core machines with 600MB of RAM."

2x400m = 800m just for the heap of the map and reduce phases, not counting the other memory that the jvm will need.  io buffer sizes aren't adjusted downward either, so you're likely looking at a swapping + spills = death scenario.  slowstart set to 1 is going to be pretty much required.