-HiveServer 2 encryption performance (takes 2.3x more time)
Arup Malakar 2013-08-19, 18:52
With HIVE-4911 hive server 2 now supports encryption for thrift
transport. The quality of protection (QoP) could be set
in hive-site.xml to either of auth, auth-int and auth-conf. Of these
auth-conf enables both encryption as well as integrity check.
In my testing I have observed that with auth-conf the amount of time taken
to transfer data is 2.3 times the time it takes
without encryption. In my test I have a table of size 1GB, and I did
"select * " on the table using the jdbc driver once with
encryption and once without encryption.
No encryption: ~9 minutes
Encryption: ~20 minutes
I was wondering if anyone has experience with SASL encryption, if it is
possible to tune any JVM/SASL settings to bring down this time.
I am also interested in understanding if it is advisable to use a different
crypto provider than the default one that ships with the JDK.
If this much overhead is to be expected with encryption methods I would
like to know that too. I am using patched version of hive-10 with Hive
on hadoop 23/jdk 1.7.