Hive, mail # dev - Re: Review Request: DB based token store

Review Request: DB based token store

Ashutosh Chauhan 2012-11-08, 01:16

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7941/
-----------------------------------------------------------

Review request for hive.
Description
-------

DB based token store
This addresses bug HIVE-3255.
   https://issues.apache.org/jira/browse/HIVE-3255
Diffs
-----

  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1389685
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1389685
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1389685
  trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java PRE-CREATION
  trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java PRE-CREATION
  trunk/metastore/src/model/package.jdo 1389685
  trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 1389685
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java PRE-CREATION
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 1389685
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1389685
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 1389685
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 1389685
  trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java PRE-CREATION
  trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1389685

Diff: https://reviews.apache.org/r/7941/diff/
Testing
-------

Includes unit test
Thanks,

Ashutosh Chauhan

Re: Review Request: DB based token store

Ashutosh Chauhan 2012-11-12, 22:54

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7941/
-----------------------------------------------------------

(Updated Nov. 12, 2012, 10:54 p.m.)
Review request for hive.
Changes
-------

Updated patch on latest trunk.
Description
-------

DB based token store
This addresses bug HIVE-3255.
   https://issues.apache.org/jira/browse/HIVE-3255
Diffs (updated)
-----

  trunk/metastore/scripts/upgrade/derby/012-HIVE-3255.derby.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql 1408480
  trunk/metastore/scripts/upgrade/mysql/012-HIVE-3255.mysql.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql 1408480
  trunk/metastore/scripts/upgrade/oracle/012-HIVE-3255.oracle.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/postgres/012-HIVE-3255.postgres.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql 1408480
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1408480
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1408480
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1408480
  trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java PRE-CREATION
  trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java PRE-CREATION
  trunk/metastore/src/model/package.jdo 1408480
  trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 1408480
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java PRE-CREATION
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 1408480
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1408480
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 1408480
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 1408480
  trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java PRE-CREATION
  trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1408480

Diff: https://reviews.apache.org/r/7941/diff/
Testing
-------

Includes unit test
Thanks,

Ashutosh Chauhan

Re: Review Request: DB based token store

Prasad Mujumdar 2012-11-13, 08:22

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7941/#review13387
-----------------------------------------------------------
The patch looks fine to me.
Just wondering if the new table definitions should also be added to hive-schema-0.10.0.XXX.sql files.

trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql
<https://reviews.apache.org/r/7941/#comment28694>

It should include 010-HIVE-3649.oracle.sql as well ?
- Prasad Mujumdar
On Nov. 12, 2012, 10:54 p.m., Ashutosh Chauhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7941/
> -----------------------------------------------------------
>
> (Updated Nov. 12, 2012, 10:54 p.m.)
>
>
> Review request for hive.
>
>
> Description
> -------
>
> DB based token store
>
>
> This addresses bug HIVE-3255.
> https://issues.apache.org/jira/browse/HIVE-3255
>
>
> Diffs
> -----
>
> trunk/metastore/scripts/upgrade/derby/012-HIVE-3255.derby.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql 1408480
> trunk/metastore/scripts/upgrade/mysql/012-HIVE-3255.mysql.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql 1408480
> trunk/metastore/scripts/upgrade/oracle/012-HIVE-3255.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/012-HIVE-3255.postgres.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1408480
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java PRE-CREATION
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java PRE-CREATION
> trunk/metastore/src/model/package.jdo 1408480
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java PRE-CREATION
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 1408480
> trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java PRE-CREATION
> trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1408480
>
> Diff: https://reviews.apache.org/r/7941/diff/
>
>
> Testing
> -------
>
> Includes unit test
>
>
> Thanks,
>
> Ashutosh Chauhan
>
>

Re: Review Request: DB based token store

Ashutosh Chauhan 2012-11-13, 08:45

> On Nov. 13, 2012, 8:22 a.m., Prasad Mujumdar wrote:
> > The patch looks fine to me.
> > Just wondering if the new table definitions should also be added to hive-schema-0.10.0.XXX.sql files.
> >

Yeah definitely. My plan is to generate ddl files after all the schema changes are in. I guess at this point only Shreepadma's work is left which deals with schema upgrade. Once thats get in, then in a separate jira I will generate all the corresponding hive-0.10.0.XXX.sql files.
> On Nov. 13, 2012, 8:22 a.m., Prasad Mujumdar wrote:
> > trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql, line 4
> > <https://reviews.apache.org/r/7941/diff/2/?file=188891#file188891line4>
> >
> > It should include 010-HIVE-3649.oracle.sql as well ?

Yeah, missed that one. Will add that in.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7941/#review13387
-----------------------------------------------------------
On Nov. 12, 2012, 10:54 p.m., Ashutosh Chauhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7941/
> -----------------------------------------------------------
>
> (Updated Nov. 12, 2012, 10:54 p.m.)
>
>
> Review request for hive.
>
>
> Description
> -------
>
> DB based token store
>
>
> This addresses bug HIVE-3255.
> https://issues.apache.org/jira/browse/HIVE-3255
>
>
> Diffs
> -----
>
> trunk/metastore/scripts/upgrade/derby/012-HIVE-3255.derby.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql 1408480
> trunk/metastore/scripts/upgrade/mysql/012-HIVE-3255.mysql.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql 1408480
> trunk/metastore/scripts/upgrade/oracle/012-HIVE-3255.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/012-HIVE-3255.postgres.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1408480
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java PRE-CREATION
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java PRE-CREATION
> trunk/metastore/src/model/package.jdo 1408480
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java PRE-CREATION
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 1408480
> trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java PRE-CREATION
> trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1408480
>
> Diff: https://reviews.apache.org/r/7941/diff/
>
>
> Testing
> -------
>
> Includes unit test
>
>
> Thanks,
>
> Ashutosh Chauhan
>
>

Re: Review Request: DB based token store

Ashutosh Chauhan 2012-11-13, 08:45

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7941/
-----------------------------------------------------------

(Updated Nov. 13, 2012, 8:45 a.m.)
Review request for hive and Carl Steinbach.
Description
-------

DB based token store
This addresses bug HIVE-3255.
   https://issues.apache.org/jira/browse/HIVE-3255
Diffs
-----

  trunk/metastore/scripts/upgrade/derby/012-HIVE-3255.derby.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql 1408480
  trunk/metastore/scripts/upgrade/mysql/012-HIVE-3255.mysql.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql 1408480
  trunk/metastore/scripts/upgrade/oracle/012-HIVE-3255.oracle.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/postgres/012-HIVE-3255.postgres.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql 1408480
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1408480
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1408480
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1408480
  trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java PRE-CREATION
  trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java PRE-CREATION
  trunk/metastore/src/model/package.jdo 1408480
  trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 1408480
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java PRE-CREATION
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 1408480
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1408480
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 1408480
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 1408480
  trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java PRE-CREATION
  trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1408480

Diff: https://reviews.apache.org/r/7941/diff/
Testing
-------

Includes unit test
Thanks,

Ashutosh Chauhan

Re: Review Request: DB based token store

Mark Grover 2012-11-15, 06:35

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7941/#review13461
-----------------------------------------------------------

trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql
<https://reviews.apache.org/r/7941/#comment28833>

   Are the prefix numbers there to keep things sorted in the upgrade/<database> directory?

   If so, (I am being nitpicky) why use 12 instead of 11? Moreover, you are using 011 as prefix in oracle directory while 012 in mysql. Should we try to be consistent?

trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql
<https://reviews.apache.org/r/7941/#comment28834>

   Fix filename to have upgrade (instead of upgrade):-)

trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
<https://reviews.apache.org/r/7941/#comment28835>

   Nitpicky: The same code to query for the token appears in addToken(), removeToken(), getToken(). Should we consider refactoring it?

trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java
<https://reviews.apache.org/r/7941/#comment28836>

   Any particular reason why this is not abstract as well?
- Mark Grover
On Nov. 13, 2012, 8:45 a.m., Ashutosh Chauhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7941/
> -----------------------------------------------------------
>
> (Updated Nov. 13, 2012, 8:45 a.m.)
>
>
> Review request for hive and Carl Steinbach.
>
>
> Description
> -------
>
> DB based token store
>
>
> This addresses bug HIVE-3255.
> https://issues.apache.org/jira/browse/HIVE-3255
>
>
> Diffs
> -----
>
> trunk/metastore/scripts/upgrade/derby/012-HIVE-3255.derby.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql 1408480
> trunk/metastore/scripts/upgrade/mysql/012-HIVE-3255.mysql.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql 1408480
> trunk/metastore/scripts/upgrade/oracle/012-HIVE-3255.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/012-HIVE-3255.postgres.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1408480
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java PRE-CREATION
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java PRE-CREATION
> trunk/metastore/src/model/package.jdo 1408480
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java PRE-CREATION
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 1408480
> trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java PRE-CREATION
> trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1408480
>
> Diff: https://reviews.apache.org/r/7941/diff/
>
>
> Testing
> -------
>
> Includes unit test
>
>
> Thanks,
>
> Ashutosh Chauhan
>
>

Re: Review Request: DB based token store

Ashutosh Chauhan 2012-11-15, 18:15

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7941/#review13476
-----------------------------------------------------------

trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
<https://reviews.apache.org/r/7941/#comment28882>

Yeah.. will refactor that.
- Ashutosh Chauhan
On Nov. 13, 2012, 8:45 a.m., Ashutosh Chauhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7941/
> -----------------------------------------------------------
>
> (Updated Nov. 13, 2012, 8:45 a.m.)
>
>
> Review request for hive and Carl Steinbach.
>
>
> Description
> -------
>
> DB based token store
>
>
> This addresses bug HIVE-3255.
> https://issues.apache.org/jira/browse/HIVE-3255
>
>
> Diffs
> -----
>
> trunk/metastore/scripts/upgrade/derby/012-HIVE-3255.derby.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql 1408480
> trunk/metastore/scripts/upgrade/mysql/012-HIVE-3255.mysql.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql 1408480
> trunk/metastore/scripts/upgrade/oracle/012-HIVE-3255.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/012-HIVE-3255.postgres.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1408480
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java PRE-CREATION
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java PRE-CREATION
> trunk/metastore/src/model/package.jdo 1408480
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java PRE-CREATION
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 1408480
> trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java PRE-CREATION
> trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1408480
>
> Diff: https://reviews.apache.org/r/7941/diff/
>
>
> Testing
> -------
>
> Includes unit test
>
>
> Thanks,
>
> Ashutosh Chauhan
>
>

Re: Review Request: DB based token store

Ashutosh Chauhan 2012-11-15, 18:17

> On Nov. 15, 2012, 6:35 a.m., Mark Grover wrote:
> > trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql, line 4
> > <https://reviews.apache.org/r/7941/diff/2/?file=188889#file188889line4>
> >
> > Are the prefix numbers there to keep things sorted in the upgrade/<database> directory?
> >
> > If so, (I am being nitpicky) why use 12 instead of 11? Moreover, you are using 011 as prefix in oracle directory while 012 in mysql. Should we try to be consistent?

Because when I generated the patch, there wasn't 11, but now there is. https://issues.apache.org/jira/browse/HIVE-3704
> On Nov. 15, 2012, 6:35 a.m., Mark Grover wrote:
> > trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql, line 1
> > <https://reviews.apache.org/r/7941/diff/2/?file=188891#file188891line1>
> >
> > Fix filename to have upgrade (instead of upgrade):-)

Will do.
> On Nov. 15, 2012, 6:35 a.m., Mark Grover wrote:
> > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java, line 430
> > <https://reviews.apache.org/r/7941/diff/2/?file=188896#file188896line430>
> >
> > Any particular reason why this is not abstract as well?

That doesn't make a difference. Its a method in interface. But for consistency, I will add abstract in there.
> On Nov. 15, 2012, 6:35 a.m., Mark Grover wrote:
> > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java, line 5310
> > <https://reviews.apache.org/r/7941/diff/2/?file=188895#file188895line5310>
> >
> > Nitpicky: The same code to query for the token appears in addToken(), removeToken(), getToken(). Should we consider refactoring it?

Yeah, make sense. Will do.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7941/#review13461
-----------------------------------------------------------
On Nov. 13, 2012, 8:45 a.m., Ashutosh Chauhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7941/
> -----------------------------------------------------------
>
> (Updated Nov. 13, 2012, 8:45 a.m.)
>
>
> Review request for hive and Carl Steinbach.
>
>
> Description
> -------
>
> DB based token store
>
>
> This addresses bug HIVE-3255.
> https://issues.apache.org/jira/browse/HIVE-3255
>
>
> Diffs
> -----
>
> trunk/metastore/scripts/upgrade/derby/012-HIVE-3255.derby.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql 1408480
> trunk/metastore/scripts/upgrade/mysql/012-HIVE-3255.mysql.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql 1408480
> trunk/metastore/scripts/upgrade/oracle/012-HIVE-3255.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/oracle/upgrate-0.9.0-to-0.10.0.oracle.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/012-HIVE-3255.postgres.sql PRE-CREATION
> trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1408480
> trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1408480
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java PRE-CREATION
> trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java PRE-CREATION
> trunk/metastore/src/model/package.jdo 1408480
> trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java PRE-CREATION
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 1408480
> trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1408480

Re: Review Request: DB based token store

Ashutosh Chauhan 2012-11-15, 21:52

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7941/
-----------------------------------------------------------

(Updated Nov. 15, 2012, 9:52 p.m.)
Review request for hive and Carl Steinbach.
Changes
-------

Updated patch based on Mark and Prasad's review comments.
Description
-------

DB based token store
This addresses bug HIVE-3255.
   https://issues.apache.org/jira/browse/HIVE-3255
Diffs (updated)
-----

  trunk/metastore/scripts/upgrade/derby/012-HIVE-3255.derby.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql 1409909
  trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql 1409909
  trunk/metastore/scripts/upgrade/mysql/012-HIVE-3255.mysql.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql 1409909
  trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql 1409909
  trunk/metastore/scripts/upgrade/oracle/012-HIVE-3255.oracle.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/oracle/hive-schema-0.10.0.oracle.sql 1409909
  trunk/metastore/scripts/upgrade/oracle/upgrade-0.9.0-to-0.10.0.oracle.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/postgres/012-HIVE-3255.postgres.sql PRE-CREATION
  trunk/metastore/scripts/upgrade/postgres/hive-schema-0.10.0.postgres.sql 1409909
  trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql 1409909
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1409909
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1409909
  trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1409909
  trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MDelegationToken.java PRE-CREATION
  trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MMasterKey.java PRE-CREATION
  trunk/metastore/src/model/package.jdo 1409909
  trunk/metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java 1409909
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DBTokenStore.java PRE-CREATION
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/DelegationTokenStore.java 1409909
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java 1409909
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java 1409909
  trunk/shims/src/common-secure/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java 1409909
  trunk/shims/src/common-secure/test/org/apache/hadoop/hive/thrift/TestDBTokenStore.java PRE-CREATION
  trunk/shims/src/common/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 1409909

Diff: https://reviews.apache.org/r/7941/diff/
Testing
-------

Includes unit test
Thanks,

Ashutosh Chauhan