Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # dev - Single Mapper - HIVE 0.11


+
Gourav Sengupta 2013-10-09, 16:58
Copy link to this message
-
Re: Single Mapper - HIVE 0.11
Nitin Pawar 2013-10-09, 17:03
whats the size of the table? (in GBs? )

Whats the max and min split sizes have you provied?
On Wed, Oct 9, 2013 at 10:28 PM, Gourav Sengupta <[EMAIL PROTECTED]>wrote:

> Hi,
>
> I am trying to run a join using two tables stored in ORC file format.
>
> The first table has 34 million records and the second has around 300,000
> records.
>
> Setting "set hive.auto.convert.join=true" makes the entire query run via a
> single mapper.
> In case I am setting "set hive.auto.convert.join=false" then there are two
> mappers first one reads the second table and then the entire large table
> goes through the second mapper.
>
> Is there something that I am doing wrong because there are three nodes in
> the HADOOP cluster currently and I was expecting that at least 6 mappers
> should have been used.
>
> Thanks and Regards,
> Gourav
>

--
Nitin Pawar
+
Prasanth Jayachandran 2013-10-09, 17:22
+
Gourav Sengupta 2013-10-10, 08:16
+
Gourav Sengupta 2013-10-11, 08:42
+
Prasanth Jayachandran 2013-10-11, 20:13