Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Review Request 24289: MetadataUpdater: provide a mechanism to edit the statistics of a column in a table (or a partition of a table)


Copy link to this message
-
Review Request 24289: MetadataUpdater: provide a mechanism to edit the statistics of a column in a table (or a partition of a table)

This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24289/

Review request for hive.
Repository: hive-git
Description

This patch provides ability to update certain stats without scanning any data or without "hacking the backend db". It helps (esp for CBO work) to set up unit tests quickly and verify both cbo and the stats subsystem. It also helps when experimenting with the system if you're just trying out hive/hadoop on a small cluster. Finally it gives you a quick and clean way to fix things when something went wrong wrt stats in your environment.
Usage:
ALTER TABLE table_name PARTITION partition_spec UPDATE STATISTICS FOR COLUMN col_name SET col_statistics
For example,
ALTER TABLE src_x_int UPDATE STATISTICS FOR COLUMN key SET ('numDVs'='101','highValue'='10001.0');
ALTER TABLE src_p PARTITION(partitionId=1) UPDATE STATISTICS FOR COLUMN key SET ('numDVs'='100','avgColLen'='1.0001');
Diffs

  cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 3cdedba
  metastore/bin/.gitignore 0dd9890
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 4c9a597
  metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java c3e2820
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java 89c31dc
  metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java 44bbab5
  ql/.gitignore 916e17c
  ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsUpdateTask.java PRE-CREATION
  ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java 24dfed1
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java e83bc17
  ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java 4300145
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 67a3aa7
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g ab1188a
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 51838ae
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 268920a
  ql/src/java/org/apache/hadoop/hive/ql/plan/ColumnStatsUpdateWork.java PRE-CREATION
  ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java 79d9d16

Diff: https://reviews.apache.org/r/24289/diff/
Testing
Thanks,

pengcheng xiong