Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> HiveServer 2 encryption performance (takes 2.3x more time)


Copy link to this message
-
HiveServer 2 encryption performance (takes 2.3x more time)
Hi,

With HIVE-4911[1] hive server 2 now supports encryption for thrift
transport. The quality of protection (QoP) could be set
in hive-site.xml to either of auth, auth-int and auth-conf. Of these
auth-conf enables both encryption as well as integrity check.
In my testing I have observed that with auth-conf the amount of time taken
to transfer data  is 2.3 times the time it takes
without encryption. In my test I have a table of size 1GB, and I did
"select * " on the table using the jdbc driver once with
encryption and once without encryption.

No encryption: ~9 minutes
Encryption:  ~20 minutes

I was wondering if anyone has experience with SASL encryption, if it is
possible to tune any JVM/SASL settings to bring down this time.
I am also interested in understanding if it is advisable to use a different
crypto provider than the default one that ships with the JDK.
If this much overhead is to be expected with encryption methods I would
like to know that too. I am using patched version of hive-10 with Hive
Server 2
on hadoop 23/jdk 1.7.

1. https://issues.apache.org/jira/browse/HIVE-4911

Thanks,
Arup Malakar