Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> how may map-reduce needed in a hive query


Copy link to this message
-
Re: how may map-reduce needed in a hive query
you can run explain extended (your query) to get more details
On Wed, Jan 23, 2013 at 9:15 AM, Richard <[EMAIL PROTECTED]> wrote:

> I am wondering how to determine the number of map-reduce for a hive query.
>
> for example, the following query
>
> select
> sum(c1),
> sum(c2),
> k1
> from
> {
> select transform(*) using 'mymapper'  as c1, c2, k1
> from t1
> } a group by k1;
>
> when i run this query, it takes two map-reduce, but I expect it to take
> only 1.
> in the map stage, using 'mymapper' as the mapper, then shuffle the mapper
> output by k1 and perform sum reduce in the reducer.
>
> so why hive takes 2 map-reduce?
>
>
>
--
Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB