Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - how may map-reduce needed in a hive query


+
Richard 2013-01-23, 03:45
+
Richard 2013-01-23, 05:54
+
Nitin Pawar 2013-01-23, 06:05
Copy link to this message
-
Re: how may map-reduce needed in a hive query
Nitin Pawar 2013-01-23, 04:07
you can run explain extended (your query) to get more details
On Wed, Jan 23, 2013 at 9:15 AM, Richard <[EMAIL PROTECTED]> wrote:

> I am wondering how to determine the number of map-reduce for a hive query.
>
> for example, the following query
>
> select
> sum(c1),
> sum(c2),
> k1
> from
> {
> select transform(*) using 'mymapper'  as c1, c2, k1
> from t1
> } a group by k1;
>
> when i run this query, it takes two map-reduce, but I expect it to take
> only 1.
> in the map stage, using 'mymapper' as the mapper, then shuffle the mapper
> output by k1 and perform sum reduce in the reducer.
>
> so why hive takes 2 map-reduce?
>
>
>
--
Nitin Pawar