Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - Re: Review Request 12824: [HIVE-4911] Enable QOP configuration for Hive Server 2 thrift transport


Copy link to this message
-
Re: Review Request 12824: [HIVE-4911] Enable QOP configuration for Hive Server 2 thrift transport
Arup Malakar 2013-07-24, 16:52


> On July 23, 2013, 9:48 p.m., Thejas Nair wrote:
> > jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java, line 142
> > <https://reviews.apache.org/r/12824/diff/1/?file=324969#file324969line142>
> >
> >     the HIVE_AUTH_TYPE env variable is called "auth".
> >     Should we use something more descriptive like "sasl.qop" as the variable that sets the QOP level.
> >
>
> Arup Malakar wrote:
>     I am totally agree that a different key name should be used for qop settings. As the current HIVE_AUTH_TYPE configuration key is overloaded. Original idea was to clean up the configuration keys which is being taken care of in: https://issues.apache.org/jira/browse/HIVE-4232. Once the auth params are taken care of, I had plans of introducing a new parameter called qop which would be used to configure the QoP alone. But since HIVE-4232 is not yet committed, I ended up using the HIVE_AUTH_TYPE. I can rebase if HIVE-4232 goes in.

I am totally agree that a different key name should be used for qop settings. As the current HIVE_AUTH_TYPE configuration key is overloaded. Original idea was to clean up the configuration keys which is being taken care of in: https://issues.apache.org/jira/browse/HIVE-4232. Once the auth params are taken care of, I had plans of introducing a new parameter called qop which would be used to configure the QoP alone. But since HIVE-4232 is not yet committed, I ended up using the HIVE_AUTH_TYPE. I can rebase if HIVE-4232 goes in.
> On July 23, 2013, 9:48 p.m., Thejas Nair wrote:
> > jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java, line 142
> > <https://reviews.apache.org/r/12824/diff/1/?file=324969#file324969line142>
> >
> >     the HIVE_AUTH_TYPE env variable is called "auth".
> >     Should we use something more descriptive like "sasl.qop" as the variable that sets the QOP level.
> >
>
> Arup Malakar wrote:
>     I am totally agree that a different key name should be used for qop settings. As the current HIVE_AUTH_TYPE configuration key is overloaded. Original idea was to clean up the configuration keys which is being taken care of in: https://issues.apache.org/jira/browse/HIVE-4232. Once the auth params are taken care of, I had plans of introducing a new parameter called qop which would be used to configure the QoP alone. But since HIVE-4232 is not yet committed, I ended up using the HIVE_AUTH_TYPE. I can rebase if HIVE-4232 goes in.

I am totally agree that a different key name should be used for qop settings. As the current HIVE_AUTH_TYPE configuration key is overloaded. Original idea was to clean up the configuration keys which is being taken care of in: https://issues.apache.org/jira/browse/HIVE-4232. Once the auth params are taken care of, I had plans of introducing a new parameter called qop which would be used to configure the QoP alone. But since HIVE-4232 is not yet committed, I ended up using the HIVE_AUTH_TYPE. I can rebase if HIVE-4232 goes in.
> On July 23, 2013, 9:48 p.m., Thejas Nair wrote:
> > shims/src/common-secure/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java, line 111
> > <https://reviews.apache.org/r/12824/diff/1/?file=324973#file324973line111>
> >
> >     This function is called from hive metastore client. Using SaslRpcServer.SASL_PROPS here means that setting hadoop.rpc.protection will determine the QOP level, if we make a call to SaslRpcServer.init(conf) from anywhere in the code. But that function is not being called.
> >    
> >     I think it makes sense to use hadoop.rpc.protection for metastore QOP, since metastore usually not exposed 'outside' the cluster unlike hive server2. It is often viewed as something 'inside the cluster'.
> >    
> >     Should we change this function to take in a configuration object and use that to call SaslRpcServer.init(conf) ?

The current createClientTransport method (without this patch) uses SaslRpcServer.SASL_PROPS too, but it doesn't call SaslRpcServer.init(conf) so I assumed SaslRpcServer.init(conf) is being called before reaching this method. But looking at https://issues.apache.org/jira/browse/HIVE-4232 I realized that this is indeed a bug in current code.

Rather than doing init() in createTransportFactory() and createClientTransport() I removed the default method that uses SaslRpcServer.SASL_PROPS. Both these methods now only takes Map<String, String>. In case of both metastore client/server the code gets the Sasl propeties from MetaStoreUtils.getMetaStoreSaslProperties(conf) and passes it to the methods in HadoopThriftAuthBridge20S.
Reasons:
1. We could remove the redundant methods that defaults to SaslRpcServer.SASL_PROPS
2. Changing meta store to use different configuration can be easily accomplished by modifying in only one place MetaStoreUtils.getMetaStoreSaslProperties(conf)

Let me know what you think of it.

I have a question though, is it okay to access SaslRpcSercer.SASL_PROPS from MetaStoreUtils class, I mean compiling with hadoop 20 wise?
- Arup
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12824/#review23711
On July 24, 2013, 4:43 p.m., Arup Malakar wrote: