Richard 2013-01-23, 03:45
Richard 2013-01-23, 05:54
Nitin Pawar 2013-01-23, 06:05
-Re: how may map-reduce needed in a hive query
Nitin Pawar 2013-01-23, 04:07
you can run explain extended (your query) to get more details
On Wed, Jan 23, 2013 at 9:15 AM, Richard <[EMAIL PROTECTED]> wrote:
> I am wondering how to determine the number of map-reduce for a hive query.
> for example, the following query
> select transform(*) using 'mymapper' as c1, c2, k1
> from t1
> } a group by k1;
> when i run this query, it takes two map-reduce, but I expect it to take
> only 1.
> in the map stage, using 'mymapper' as the mapper, then shuffle the mapper
> output by k1 and perform sum reduce in the reducer.
> so why hive takes 2 map-reduce?