Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Review Request 24203: PIG-4091: Predicate pushdown for ORC


Copy link to this message
-
Review Request 24203: PIG-4091: Predicate pushdown for ORC

This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24203/

Review request for pig and Daniel Dai.
Bugs: PIG-4091
    https://issues.apache.org/jira/browse/PIG-4091
Repository: pig
Description

Has the implementation of Predicate Pushdown as a hint.

Pending items to be addressed in separate jira.
   - e2e tests. Will port FilterEq and FilterBoolean from nightly.conf to OrcStorage
   - boolean, biginteger, bigdecimal, datetime datatypes pushdown

JIRA is being reindexed and not available. Will upload patch there once jira is available
Diffs

  http://svn.apache.org/repos/asf/pig/trunk/ivy.xml 1615182
  http://svn.apache.org/repos/asf/pig/trunk/ivy/libraries.properties 1615182
  http://svn.apache.org/repos/asf/pig/trunk/shims/test/hadoop20/org/apache/pig/test/MiniCluster.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/shims/test/hadoop20/org/apache/pig/test/TezMiniCluster.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/shims/test/hadoop23/org/apache/pig/test/MiniCluster.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/shims/test/hadoop23/org/apache/pig/test/TezMiniCluster.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/Expression.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/LoadPredicatePushdown.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/Main.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/OrcStorage.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/FilterExtractor.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/PartitionFilterExtractor.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/PredicatePushDownFilterExtractor.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/optimizer/LogicalPlanOptimizer.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/PartitionFilterOptimizer.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/logical/rules/PredicatePushdownOptimizer.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/newplan/optimizer/PlanOptimizer.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/builtin/TestOrcStorage.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/builtin/TestOrcStoragePushdown.java PRE-CREATION
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/MiniGenericCluster.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestNewPartitionFilterPushDown.java 1615182
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/Util.java 1615182

Diff: https://reviews.apache.org/r/24203/diff/
Testing

Unit tests added. Will add e2e in a separate jira.
Thanks,

Rohini Palaniswamy