Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # dev >> Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs


+
Yin Huai 2013-07-21, 05:37
+
Yin Huai 2013-07-22, 04:19
+
Gunther Hagleitner 2013-07-23, 02:24
+
Yin Huai 2013-07-24, 16:52
Copy link to this message
-
Re: Review Request 12795: [HIVE-4827] Merge a Map-only job to its following MapReduce job with multiple inputs

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12795/#review23671
-----------------------------------------------------------

ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java
<https://reviews.apache.org/r/12795/#comment47569>

    Don't we still need the copyright?

ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
<https://reviews.apache.org/r/12795/#comment47572>

    if you change the input to an array or collection, you don't have to duplicate code below (use a loop)

ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
<https://reviews.apache.org/r/12795/#comment47573>

    Optional, but: This name (HIVECONVERTJOINNOCONDITANDSOONANDSOFORTH) is horrible. Can you just replace with HIVE_CONDITIONAL_TASK_THRESHOLD? We might have to keep both external names for backwards compat.

ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
<https://reviews.apache.org/r/12795/#comment47576>

    You're dealing with a single key one list of values here. Why not just use those instead of messing around with entries and iterators. This makes it hard to read.

ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
<https://reviews.apache.org/r/12795/#comment47577>

    Why is it better to throw exception here than simply return?

ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
<https://reviews.apache.org/r/12795/#comment47578>

    Why is this? Should work regardless, no?

ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java
<https://reviews.apache.org/r/12795/#comment47579>

    see below, utils would be better

ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java
<https://reviews.apache.org/r/12795/#comment47574>

    Can you put this into the GenMapRedUtils? This is planning stuff not execution stuff.
- Gunther Hagleitner
On July 22, 2013, 4:19 a.m., Yin Huai wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12795/
> -----------------------------------------------------------
>
> (Updated July 22, 2013, 4:19 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-4827
>     https://issues.apache.org/jira/browse/HIVE-4827
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/HIVE-4827
>
>
> Diffs
> -----
>
>   ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorUtils.java 66b84ff
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/CommonJoinTaskDispatcher.java f98878c
>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 7cbb1ff
>   ql/src/test/queries/clientpositive/correlationoptimizer7.q 9b18972
>   ql/src/test/queries/clientpositive/multiMapJoin2.q PRE-CREATION
>   ql/src/test/results/clientpositive/auto_join33.q.out 8fc0e84
>   ql/src/test/results/clientpositive/correlationoptimizer1.q.out db3bd78
>   ql/src/test/results/clientpositive/correlationoptimizer3.q.out cebddff
>   ql/src/test/results/clientpositive/correlationoptimizer4.q.out 285a54f
>   ql/src/test/results/clientpositive/correlationoptimizer6.q.out c40a786
>   ql/src/test/results/clientpositive/correlationoptimizer7.q.out ea54431
>   ql/src/test/results/clientpositive/multiMapJoin1.q.out 3b3eb3f
>   ql/src/test/results/clientpositive/multiMapJoin2.q.out PRE-CREATION
>
> Diff: https://reviews.apache.org/r/12795/diff/
>
>
> Testing
> -------
>
> Running tests.
>
>
> Thanks,
>
> Yin Huai
>
>

+
Yin Huai 2013-07-23, 19:38
+
Yin Huai 2013-07-25, 22:50
+
Yin Huai 2013-07-26, 18:51
+
Yin Huai 2013-07-27, 00:21
+
Yin Huai 2013-07-30, 19:35
+
Yin Huai 2013-07-31, 01:06
+
Yin Huai 2013-07-31, 03:38