-how may map-reduce needed in a hive query
Richard 2013-01-23, 03:45
I am wondering how to determine the number of map-reduce for a hive query.
for example, the following query
select transform(*) using 'mymapper' as c1, c2, k1
} a group by k1;
when i run this query, it takes two map-reduce, but I expect it to take only 1.
in the map stage, using 'mymapper' as the mapper, then shuffle the mapper output by k1 and perform sum reduce in the reducer.
so why hive takes 2 map-reduce?
Richard 2013-01-23, 05:54
Nitin Pawar 2013-01-23, 06:05
Nitin Pawar 2013-01-23, 04:07