Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Skewed Join


Copy link to this message
-
Re: Skewed Join
can you share your query and your table definition ?
On Fri, Jun 7, 2013 at 12:12 PM, sumit ghosh <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am hit by skewed Join, my last reducer is getting same number of Reduce
> input groups/records.
> Reduce input groups                      432,446,942
> Reduce shuffle bytes                  13,012,613,275
> Reduce input records                     432,446,942
>
> Why is this happening? I have turned on skew join optimization:
> hive.optimize.skewjoin=true;
> ****hive.skewjoin.key=100000;**
> ****
> ****
> **Thanks,**
> **Sumit**
> **
>
> **
>

--
Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB