Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> how may map-reduce needed in a hive query

Copy link to this message
how may map-reduce needed in a hive query
I am wondering how to determine the number of map-reduce for a hive query.
for example, the following query
select transform(*) using 'mymapper'  as c1, c2, k1
from t1
} a group by k1;
when i run this query, it takes two map-reduce, but I expect it to take only 1.
in the map stage, using 'mymapper' as the mapper, then shuffle the mapper output by k1 and perform sum reduce in the reducer.
so why hive takes 2 map-reduce?
Richard 2013-01-23, 05:54
Nitin Pawar 2013-01-23, 06:05
Nitin Pawar 2013-01-23, 04:07