Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> HiveServer 2 encryption performance (takes 2.3x more time)


Copy link to this message
-
HiveServer 2 encryption performance (takes 2.3x more time)
Hi,

With HIVE-4911[1] hive server 2 now supports encryption for thrift
transport. The quality of protection (QoP) could be set
in hive-site.xml to either of auth, auth-int and auth-conf. Of these
auth-conf enables both encryption as well as integrity check.
In my testing I have observed that with auth-conf the amount of time taken
to transfer data  is 2.3 times the time it takes
without encryption. In my test I have a table of size 1GB, and I did
"select * " on the table using the jdbc driver once with
encryption and once without encryption.

No encryption: ~9 minutes
Encryption:  ~20 minutes

I was wondering if anyone has experience with SASL encryption, if it is
possible to tune any JVM/SASL settings to bring down this time.
I am also interested in understanding if it is advisable to use a different
crypto provider than the default one that ships with the JDK.
If this much overhead is to be expected with encryption methods I would
like to know that too. I am using patched version of hive-10 with Hive
Server 2
on hadoop 23/jdk 1.7.

1. https://issues.apache.org/jira/browse/HIVE-4911

Thanks,
Arup Malakar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB