Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Review Request 16074: HIVE-5951 improve performance of adding partitions from client


Copy link to this message
-
Re: Review Request 16074: HIVE-5951 improve performance of adding partitions from client

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16074/#review31486
-----------------------------------------------------------

metastore/if/hive_metastore.thrift
<https://reviews.apache.org/r/16074/#comment59988>

    Should this be list<bool>? I guess in current form, this implies same boolean value for all partitions passed in the request. Will be good to document.

metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
<https://reviews.apache.org/r/16074/#comment59989>

    I wonder instead we should create all MPartition and then do pm.makePersistentAll(List<MPartition>) ?

ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
<https://reviews.apache.org/r/16074/#comment59990>

    Lets pass false, otherwise server may return huge output (for large number of partitions) which is of no use to client.

ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java
<https://reviews.apache.org/r/16074/#comment59991>

    We should do this via Kryo. Probably matter for another jira though.

ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
<https://reviews.apache.org/r/16074/#comment59992>

    It its not lot of work, might make sense to do this TODO now.
- Ashutosh Chauhan
On Dec. 9, 2013, 7:36 p.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16074/
> -----------------------------------------------------------
>
> (Updated Dec. 9, 2013, 7:36 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> See JIRA. RB does not include generated code.
>
>
> Diffs
> -----
>
>   metastore/if/hive_metastore.thrift 43b3907
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 01c2626
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 65406d9
>   metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java cacfa07
>   metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 04d399f
>   metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 27ae3c4
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java 57f1e67
>   metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java c0e720f
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java d32be59
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 947b65c
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Partition.java f4476a9
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 321759b
>   ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 7443ea4
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java e97d948
>   ql/src/java/org/apache/hadoop/hive/ql/plan/AddPartitionDesc.java ff60e44
>   ql/src/test/results/clientnegative/addpart1.q.out b4be19c
>   ql/src/test/results/clientnegative/alter_view_failure4.q.out b218c19
>   ql/src/test/results/clientpositive/add_part_exist.q.out 559cb26
>   ql/src/test/results/clientpositive/add_part_multiple.q.out 7bc071e
>   ql/src/test/results/clientpositive/create_view_partitioned.q.out e90ffc7
>   ql/src/test/results/clientpositive/partitions_json.q.out deb7a1f
>
> Diff: https://reviews.apache.org/r/16074/diff/
>
>
> Testing
> -------
>
> running clidriver, some more query results will change
>
>
> Thanks,
>
> Sergey Shelukhin
>
>