Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # dev >> Single Mapper - HIVE 0.11


+
Gourav Sengupta 2013-10-09, 16:58
Copy link to this message
-
Re: Single Mapper - HIVE 0.11
whats the size of the table? (in GBs? )

Whats the max and min split sizes have you provied?
On Wed, Oct 9, 2013 at 10:28 PM, Gourav Sengupta <[EMAIL PROTECTED]>wrote:

> Hi,
>
> I am trying to run a join using two tables stored in ORC file format.
>
> The first table has 34 million records and the second has around 300,000
> records.
>
> Setting "set hive.auto.convert.join=true" makes the entire query run via a
> single mapper.
> In case I am setting "set hive.auto.convert.join=false" then there are two
> mappers first one reads the second table and then the entire large table
> goes through the second mapper.
>
> Is there something that I am doing wrong because there are three nodes in
> the HADOOP cluster currently and I was expecting that at least 6 mappers
> should have been used.
>
> Thanks and Regards,
> Gourav
>

--
Nitin Pawar
+
Prasanth Jayachandran 2013-10-09, 17:22
+
Gourav Sengupta 2013-10-10, 08:16
+
Gourav Sengupta 2013-10-11, 08:42
+
Prasanth Jayachandran 2013-10-11, 20:13
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB