Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - what does "keep 10% map, 40% reduce" mean in gridmix2's README?


Copy link to this message
-
what does "keep 10% map, 40% reduce" mean in gridmix2's README?
Nan Zhu 2012-06-14, 06:58
Hi, all

I'm using gridmix2 to test my cluster, while in its README file, there are
statements like the following:

+1) Three stage map/reduce job
+   Input:      500GB compressed (2TB uncompressed) SequenceFile
+                 (k,v) = (5 words, 100 words)
+                 hadoop-env: FIXCOMPSEQ
+     *Compute1:   keep 10% map, 40% reduce
+   Compute2:   keep 100% map, 77% reduce
+                 Input from Compute1
+     Compute3:   keep 116% map, 91% reduce
+                 Input from Compute2
+     *Motivation: Many user workloads are implemented as pipelined map/reduce
+                 jobs, including Pig workloads
Can anyone tell me what does "keep 10% map, 40% reduce" mean here?

Best,

--
Nan Zhu
School of Electronic, Information and Electrical Engineering,229
Shanghai Jiao Tong University
800,Dongchuan Road,Shanghai,China
E-Mail: [EMAIL PROTECTED]
+
Chen He 2012-06-14, 15:50
+
Nan Zhu 2012-06-14, 16:42
+
Chen He 2012-06-14, 17:09
+
gemini alex 2012-06-25, 06:50
+
Nan Zhu 2012-06-25, 09:45