Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - Review Request 14243: HIVE-5325: Implement statistics providing ORC writer and reader interfaces


Copy link to this message
-
Re: Review Request 14243: HIVE-5325: Implement statistics providing ORC writer and reader interfaces
j.prasanth.j@... 2013-10-01, 01:55

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14243/
-----------------------------------------------------------

(Updated Oct. 1, 2013, 1:55 a.m.)
Review request for hive, Ashutosh Chauhan and Owen O'Malley.
Changes
-------

Fixed a failed test case.
Bugs: HIVE-5325
    https://issues.apache.org/jira/browse/HIVE-5325
Repository: hive-git
Description
-------

HIVE-5324 adds new interfaces that can be implemented by ORC reader/writer to provide statistics. Writer provided statistics is used to update table/partition level statistics in metastore. Reader provided statistics can be used for reducer estimation, CBO etc. in the absence of metastore statistics.
Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/io/orc/BinaryColumnStatistics.java PRE-CREATION
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/ColumnStatisticsImpl.java 6268617
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcOutputFormat.java 6f8ca73
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/ReaderImpl.java e034ca0
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/StringColumnStatistics.java 72e779a
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java c0b55ce
  ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java e3eec02
  ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto edbf822
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcFile.java e6569f4
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcNullOptimization.java b93db84
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcSerDeStats.java PRE-CREATION
  ql/src/test/resources/orc-file-dump-dictionary-threshold.out 003c132
  ql/src/test/resources/orc-file-dump.out fac5326

Diff: https://reviews.apache.org/r/14243/diff/
Testing
-------

ORC related unit and qfile tests are passing.
Thanks,

Prasanth_J