Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Fwd: Map join optimization issue


Copy link to this message
-
Fwd: Map join optimization issue
Hello all,
I am trying to join two tables, the smaller being of size 4GB. When I set
hive.mapjoin.smalltable.filesize parameter above 500MB, Hive tries to
perform a local task to read the smaller file. This of-course fails since
the file size is greater and the backup common join is then run. What I do
not understand is why did Hive attempt a map join when small file size was
greater than the smalltable.filesize parameter.
~Mayuresh
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB