Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # dev - Review Request 15184: Store state of stats


+
Ashutosh Chauhan 2013-11-01, 18:03
+
Ashutosh Chauhan 2013-11-01, 18:23
+
Thejas Nair 2013-11-05, 02:04
+
Ashutosh Chauhan 2013-11-06, 16:35
+
Thejas Nair 2013-11-05, 01:56
Copy link to this message
-
Re: Review Request 15184: Store state of stats
Ashutosh Chauhan 2013-11-06, 16:35


> On Nov. 5, 2013, 1:56 a.m., Thejas Nair wrote:
> > trunk/common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java, line 94
> > <https://reviews.apache.org/r/15184/diff/1/?file=376592#file376592line94>
> >
> >     Can you add a comment describing that this is used to indicating to alter table command that the stats were populated by the StatsTask
> >

Done.
> On Nov. 5, 2013, 1:56 a.m., Thejas Nair wrote:
> > trunk/common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java, line 96
> > <https://reviews.apache.org/r/15184/diff/1/?file=376592#file376592line96>
> >
> >     Can you add a comment describing what this param can be use for ?

Done.
> On Nov. 5, 2013, 1:56 a.m., Thejas Nair wrote:
> > trunk/common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java, line 99
> > <https://reviews.apache.org/r/15184/diff/1/?file=376592#file376592line99>
> >
> >     looks like there is some extra white space.

Done.
> On Nov. 5, 2013, 1:56 a.m., Thejas Nair wrote:
> > trunk/common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java, line 103
> > <https://reviews.apache.org/r/15184/diff/1/?file=376592#file376592line103>
> >
> >     a nit - you can avoid two hash lookups, if you do a stats_acc = params.get(COLUMN_STATS_ACCURATE) and have a if(stats_acc == null){
> >     return false;
> >     } else {
> >      return stats_acc.equals(TRUE);
> >     }

Done.
> On Nov. 5, 2013, 1:56 a.m., Thejas Nair wrote:
> > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java, line 181
> > <https://reviews.apache.org/r/15184/diff/1/?file=376593#file376593line181>
> >
> >     the update*StatsFast is not getting called from the StatsTask alterTable call stack, if stats autogather config is false. But we need to invalidate the column stats irrespective of the autogather settings.
> >    
> >     In case of fast stats, if we meet the criteria for updating stats, except for the autogather settings, I think it makes sense to invalidate the fast stats also for those.
> >

Done.
> On Nov. 5, 2013, 1:56 a.m., Thejas Nair wrote:
> > trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java, line 220
> > <https://reviews.apache.org/r/15184/diff/1/?file=376596#file376596line220>
> >
> >     indentation problems in this file

Done.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15184/#review28061
-----------------------------------------------------------
On Nov. 6, 2013, 4:32 p.m., Ashutosh Chauhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15184/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2013, 4:32 p.m.)
>
>
> Review request for hive and Thejas Nair.
>
>
> Bugs: HIVE-3777
>     https://issues.apache.org/jira/browse/HIVE-3777
>
>
> Repository: hive
>
>
> Description
> -------
>
> Store state of stats.
>
>
> Diffs
> -----
>
>   trunk/common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java 1539099
>   trunk/hbase-handler/src/test/results/positive/hbase_stats.q.out 1539099
>   trunk/hbase-handler/src/test/results/positive/hbase_stats2.q.out 1539099
>   trunk/hbase-handler/src/test/results/positive/hbase_stats3.q.out 1539099
>   trunk/hbase-handler/src/test/results/positive/hbase_stats_empty_partition.q.out 1539099
>   trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1539099
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/StatsTask.java 1539099
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1539099
>   trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java 1539099
>   trunk/ql/src/test/queries/clientpositive/stats_invalidation.q PRE-CREATION
>   trunk/ql/src/test/queries/clientpositive/stats_only_null.q 1539099
+
Ashutosh Chauhan 2013-11-06, 16:32