Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc


Copy link to this message
-
Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc
Yin Huai 2013-09-19, 17:48

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14221/
-----------------------------------------------------------

(Updated Sept. 19, 2013, 5:48 p.m.)
Review request for hive.
Bugs: HIVE-4113
    https://issues.apache.org/jira/browse/HIVE-4113
Repository: hive-git
Description
-------

Modifies ColumnProjectionUtils such there are two flags. One for the column ids and one indicating whether all columns should be read. Additionally the patch updates all locations which uses the old method of empty string indicating all columns should be read.

The automatic formatter generated by ant eclipse-files is fairly aggressive so there are some unrelated import/whitespace cleanup.

This one is based on https://reviews.apache.org/r/11770/ and has been rebased to the latest trunk.
Diffs (updated)
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9f37d0c
  conf/hive-default.xml.template 545026d
  hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java 766056b
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseInputFormat.java 553446a
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatRecordReader.java 3ee6157
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/InitializeInput.java 1980ef5
  hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatPartitioned.java 577e06d
  hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestHCatLoader.java d38bb8d
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 31a52ba
  ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java ab0494e
  ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java a5a8943
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 0f29a0e
  ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveInputFormat.java 49145b7
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java cccdc1b
  ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java a83f223
  ql/src/java/org/apache/hadoop/hive/ql/io/RCFileRecordReader.java 9521060
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 50c5093
  ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/RCFileBlockMergeRecordReader.java cbdc2db
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java ed14e82
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java b97d869
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/MetadataOnlyOptimizer.java 0550bf6
  ql/src/test/org/apache/hadoop/hive/ql/io/PerformTestRCFileAndSeqFile.java fb9fca1
  ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java dd1276d
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 83c5c38
  serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java 0b3ef7b
  serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java 11f5f07
  serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarStruct.java 1335446
  serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarStructBase.java e1270cc
  serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarSerDe.java b717278
  serde/src/java/org/apache/hadoop/hive/serde2/columnar/LazyBinaryColumnarStruct.java 0317024
  serde/src/test/org/apache/hadoop/hive/serde2/TestColumnProjectionUtils.java PRE-CREATION

Diff: https://reviews.apache.org/r/14221/diff/
Testing
-------
Thanks,

Yin Huai