Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - Review Request 15561: HIVE-5771 Constant propagation optimizer for Hive


Copy link to this message
-
Review Request 15561: HIVE-5771 Constant propagation optimizer for Hive
Ted Xu 2013-11-15, 01:57

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15561/
-----------------------------------------------------------

Review request for hive.
Bugs: HIVE-5771
    https://issues.apache.org/jira/browse/HIVE-5771
Repository: hive
Description
-------

Currently there is no constant folding/propagation optimizer, all expressions are evaluated at runtime.

HIVE-2470 did a great job on evaluating constants on UDF initializing phase, however, it is still a runtime evaluation and it doesn't propagate constants from a subquery to outside.

It may reduce I/O and accelerate process if we introduce such an optimizer.
Diffs
-----

  http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnInfo.java 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagate.java PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcCtx.java PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog1.q PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog2.q PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_dp.q PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cast1.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/column_access_stats.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog1.q.out PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_dp.q.out PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/create_like_view.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_ppd.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_sort_1.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_sort_skew_1.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_empty.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_file_format.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_multiple.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_partitioned.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_update.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_stale.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_stale_partitioned.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/infer_const_type.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input18.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input23.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input26.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input38.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input6.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input_part4.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input_part6.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/insert1.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join38.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_cond_pushdown_1.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_cond_pushdown_3.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_nullsafe.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/join_view.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/lateral_view_ppd.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/literal_double.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/macro.q.out 1541857
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/mapjoin1.q.out 1541857
  http://sv