Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Review Request 14953: Pushdown join conditions


Copy link to this message
-
Re: Review Request 14953: Pushdown join conditions


> On Oct. 31, 2013, 12:34 a.m., Ashutosh Chauhan wrote:
> > ql/src/test/results/clientpositive/join_cond_pushdown_2.q.out, line 285
> > <https://reviews.apache.org/r/14953/diff/1/?file=371574#file371574line285>
> >
> >     Can this be done as follows:
> >     First MR job:
> >      Mapper reads P1,P2,P3 and partitions by partname
> >     In reducer 3-way join of P1,P2,P3 on partname, followed by post-join filter of p1.p_partkey=p2.p_partkey
> >    
> >     Second MR job:
> >     Mapper reads output of previous job and P4 and partition both sides by p_partkey and than join them on p_partkey in reducer.
> >     Seems like this will generate correct results.
> >

Yes I think this works. Nice! Will check with Vikram and John also.
So we should add an Optimization that tries to convert 2 consecutive Join Operators into 1 JoinOp followed by a FilterOp, Right?
This works only for inner joins.
- Harish
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14953/#review27840
-----------------------------------------------------------
On Oct. 29, 2013, 9:19 p.m., Harish Butani wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14953/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2013, 9:19 p.m.)
>
>
> Review request for hive, Ashutosh Chauhan and Vikram Dixit Kumaraswamy.
>
>
> Bugs: hive-5556
>     https://issues.apache.org/jira/browse/hive-5556
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Step 1 to support Alternate Join Syntax: HIVE-5555
>
> This patch also contains fixes to merging of QBJoinTrees
>
>
> Diffs
> -----
>
>   ql/src/java/org/apache/hadoop/hive/ql/parse/QBJoinTree.java 9c8cac1
>   ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java cf0c895
>   ql/src/test/org/apache/hadoop/hive/ql/parse/TestQBJoinTreeApplyPredicate.java PRE-CREATION
>   ql/src/test/queries/clientpositive/join_cond_pushdown_1.q PRE-CREATION
>   ql/src/test/queries/clientpositive/join_cond_pushdown_2.q PRE-CREATION
>   ql/src/test/results/clientpositive/auto_sortmerge_join_12.q.out 865627b
>   ql/src/test/results/clientpositive/join_cond_pushdown_1.q.out PRE-CREATION
>   ql/src/test/results/clientpositive/join_cond_pushdown_2.q.out PRE-CREATION
>
> Diff: https://reviews.apache.org/r/14953/diff/
>
>
> Testing
> -------
>
> ran all join .q files
> added join_cond_pushdown_1.q, join_cond_pushdown_2.q .q tests
> added TestQBJoinTreeApplyPredicate unit test to test pushdown functionality
>
>
> Thanks,
>
> Harish Butani
>
>