Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Cluster Tuning


+
Juan P. 2011-07-07, 20:29
+
Joey Echeverria 2011-07-07, 20:44
+
Esteban Gutierrez 2011-07-07, 20:53
+
Ceriasmex 2011-07-07, 22:25
+
Juan P. 2011-07-08, 14:41
+
Juan P. 2011-07-08, 15:25
+
Robert Evans 2011-07-08, 16:11
+
Joey Echeverria 2011-07-08, 16:14
+
Bharath Mundlapudi 2011-07-08, 19:21
+
Juan P. 2011-07-11, 16:28
+
Juan P. 2011-07-11, 16:41
Copy link to this message
-
Re: Cluster Tuning
Allen Wittenauer 2011-07-11, 19:41

On Jul 11, 2011, at 9:28 AM, Juan P. wrote:
>
> *  <property>*
> *    <name>mapred.child.java.opts</name>*
> *    <value>-Xmx400m</value>*
> *  </property>*

"Single core machines with 600MB of RAM."

2x400m = 800m just for the heap of the map and reduce phases, not counting the other memory that the jvm will need.  io buffer sizes aren't adjusted downward either, so you're likely looking at a swapping + spills = death scenario.  slowstart set to 1 is going to be pretty much required.
+
Juan P. 2011-07-11, 21:13
+
Steve Loughran 2011-07-15, 20:51