Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - Review Request 15561: HIVE-5771 Constant propagation optimizer for Hive


Copy link to this message
-
Re: Review Request 15561: HIVE-5771 Constant propagation optimizer for Hive
Ted Xu 2014-01-18, 14:54

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15561/
-----------------------------------------------------------

(Updated Jan. 18, 2014, 2:54 p.m.)
Review request for hive.
Changes
-------

Fixed unit test failures.
Bugs: HIVE-5771
    https://issues.apache.org/jira/browse/HIVE-5771
Repository: hive
Description
-------

Currently there is no constant folding/propagation optimizer, all expressions are evaluated at runtime.

HIVE-2470 did a great job on evaluating constants on UDF initializing phase, however, it is still a runtime evaluation and it doesn't propagate constants from a subquery to outside.

It may reduce I/O and accelerate process if we introduce such an optimizer.
Diffs (updated)
-----

  http://svn.apache.org/repos/asf/hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1559345
  http://svn.apache.org/repos/asf/hive/trunk/contrib/src/test/results/clientpositive/udf_example_add.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/contrib/src/test/results/clientpositive/udf_example_format.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/hbase-handler/src/test/results/positive/hbase_ppd_key_range.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/hbase-handler/src/test/results/positive/hbase_pushdown.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/hbase-handler/src/test/results/positive/ppd_key_ranges.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnInfo.java 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagate.java PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcCtx.java PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConstantPropagateProcFactory.java PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/alter_partition_coltype.q 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/annotate_stats_filter.q 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/annotate_stats_part.q 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/annotate_stats_select.q 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog1.q PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog2.q PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_dp.q PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/constprog_type.q PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/decimal_udf.q 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/pcr.q 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cast1.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/cluster.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/column_access_stats.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog1.q.out PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog2.q.out PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_dp.q.out PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/constprog_type.q.out PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_ppd.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_sort_1.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/groupby_sort_skew_1.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_empty.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_file_format.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_multiple.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_partitioned.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_auto_update.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_stale.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/index_stale_partitioned.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/infer_const_type.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input18.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input23.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input26.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/input38.q.out 1559345
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/client