Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # dev - Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc


+
Yin Huai 2013-09-18, 22:47
+
Yin Huai 2013-09-19, 02:55
+
Yin Huai 2013-09-19, 17:48
+
Yin Huai 2013-09-19, 21:05
+
Ashutosh Chauhan 2013-09-19, 18:49
+
Yin Huai 2013-09-20, 03:18
+
Yin Huai 2013-09-20, 14:24
+
Yin Huai 2013-09-20, 15:48
+
Yin Huai 2013-09-20, 21:07
+
Yin Huai 2013-09-21, 01:46
+
Yin Huai 2013-09-21, 22:27
Copy link to this message
-
Re: Review Request 14221: HIVE-4113: Optimize select count(1) with RCFile and Orc
Yin Huai 2013-09-21, 22:48

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14221/
-----------------------------------------------------------

(Updated Sept. 21, 2013, 10:47 p.m.)
Review request for hive.
Bugs: HIVE-4113
    https://issues.apache.org/jira/browse/HIVE-4113
Repository: hive-git
Description
-------

Modifies ColumnProjectionUtils such there are two flags. One for the column ids and one indicating whether all columns should be read. Additionally the patch updates all locations which uses the old method of empty string indicating all columns should be read.

The automatic formatter generated by ant eclipse-files is fairly aggressive so there are some unrelated import/whitespace cleanup.

This one is based on https://reviews.apache.org/r/11770/ and has been rebased to the latest trunk.
Diffs (updated)
-----

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 381bcbe
  conf/hive-default.xml.template 6531e55
  contrib/src/test/results/clientpositive/serde_typedbytes.q.out 8c22399
  contrib/src/test/results/clientpositive/serde_typedbytes2.q.out 1e4881f
  contrib/src/test/results/clientpositive/serde_typedbytes3.q.out 0186983
  contrib/src/test/results/clientpositive/serde_typedbytes5.q.out ece8e43
  contrib/src/test/results/clientpositive/udf_row_sequence.q.out f745840
  hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java 766056b
  hbase-handler/src/test/results/positive/hbase_queries.q.out 0bd55f6
  hbase-handler/src/test/results/positive/hbase_single_sourced_multi_insert.q.out 92e8175
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/HCatBaseInputFormat.java 553446a
  hcatalog/core/src/test/java/org/apache/hive/hcatalog/mapreduce/TestHCatPartitioned.java 577e06d
  hcatalog/hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestHCatLoader.java d38bb8d
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 31a52ba
  ql/src/java/org/apache/hadoop/hive/ql/exec/FetchTask.java df2ccf1
  ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java ab0494e
  ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java a5a8943
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 0f29a0e
  ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveInputFormat.java 49145b7
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java cccdc1b
  ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java a83f223
  ql/src/java/org/apache/hadoop/hive/ql/io/RCFileRecordReader.java 9521060
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 50c5093
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java ed14e82
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java 1ede6d7
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java 52e9e6b
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 2259977
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java b97d869
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/MetadataOnlyOptimizer.java 0550bf6
  ql/src/test/org/apache/hadoop/hive/ql/io/PerformTestRCFileAndSeqFile.java fb9fca1
  ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java dd1276d
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestInputOutputFormat.java 83c5c38
  ql/src/test/queries/clientpositive/binary_table_colserde.q eadf07d
  ql/src/test/results/clientpositive/auto_join0.q.out a75c01c
  ql/src/test/results/clientpositive/auto_join15.q.out 6fb0ea6
  ql/src/test/results/clientpositive/auto_join18.q.out 945af67
  ql/src/test/results/clientpositive/auto_join18_multi_distinct.q.out 500df42
  ql/src/test/results/clientpositive/auto_join20.q.out 6dd8ff7
  ql/src/test/results/clientpositive/auto_join27.q.out aac778c
  ql/src/test/results/clientpositive/auto_join30.q.out b5b313c
  ql/src/test/results/clientpositive/auto_join31.q.out ee8204f
  ql/src/test/results/clientpositive/auto_join_reordering_values.q.out 53ce112
  ql/src/test/results/clientpositive/auto_smb_mapjoin_14.q.out 2bc99fa
  ql/src/test/results/clientpositive/auto_sortmerge_join_10.q.out 0cd7734
  ql/src/test/results/clientpositive/auto_sortmerge_join_6.q.out 1274b76
  ql/src/test/results/clientpositive/auto_sortmerge_join_9.q.out 96fcd2b
  ql/src/test/results/clientpositive/binary_output_format.q.out ad245f2
  ql/src/test/results/clientpositive/binary_table_colserde.q.out 69a6c6e
  ql/src/test/results/clientpositive/bucket5.q.out 41e4a3e
  ql/src/test/results/clientpositive/bucketizedhiveinputformat.q.out e4beebc
  ql/src/test/results/clientpositive/bucketmapjoin1.q.out 122d061
  ql/src/test/results/clientpositive/bucketmapjoin2.q.out 955b8a2
  ql/src/test/results/clientpositive/bucketmapjoin3.q.out e0b86ae
  ql/src/test/results/clientpositive/bucketmapjoin4.q.out bed6a0a
  ql/src/test/results/clientpositive/bucketmapjoin5.q.out 21a47c0
  ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 2feec49
  ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out e524220
  ql/src/test/results/clientpositive/case_sensitivity.q.out 40fc7e2
  ql/src/test/results/clientpositive/cast1.q.out 2bc2803
  ql/src/test/results/clientpositive/cluster.q.out e139b60
  ql/src/test/results/clientpositive/column_access_stats.q.out 991c492
  ql/src/test/results/clientpositive/correlationoptimizer1.q.out 2b8a149
  ql/src/test/results/clientpositive/correlationoptimizer10.q.out 5886d4c
  ql/src/test/results/clientpositive/correlationoptimizer11.q.out 72439f8
  ql/src/test/results/clientpositive/correlationoptimizer12.q.out 22dfce9
  ql/src/test/results/clientpositive/correlationoptimizer13.q.out 5eed255
  ql/src/test/results/clientpositive/correlationoptimizer14.q.out 99263aa
  ql/src/test/results/clientpositive/correlationoptimizer15.q.out 76318b5
  ql/src/test/results/clientpositive/correlationoptimiz
+
Yin Huai 2013-09-22, 01:51