Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Encryption in HDFS


+
Seonyeong Bak 2013-02-26, 06:51
+
Ted Dunning 2013-02-26, 05:55
+
Ted Yu 2013-02-26, 05:24
Copy link to this message
-
Re: Encryption in HDFS
Another challenge of encrypt/decrypt is key management.
Can  you share how are this is handled in your implementation/research

2013/2/25 Seonyeong Bak <[EMAIL PROTECTED]>

> Hello, I'm a university student.
>
> I implemented AES and Triple DES with CompressionCodec in java
> cryptography architecture (JCA)
> The encryption is performed by a client node using Hadoop API.
> Map tasks read blocks from HDFS and these blocks are decrypted by each map
> tasks.
> I tested my implementation with generic HDFS.
> My cluster consists of 3 nodes (1 master node, 3 worker nodes) and each
> machines have quad core processor (i7-2600) and 4GB memory.
> A test input is 1TB text file which consists of 32 multiple text files (1
> text file is 32GB)
>
> I expected that the encryption takes much more time than generic HDFS.
> The performance does not differ significantly.
> The decryption step takes about 5-7% more than generic HDFS.
> The encryption step takes about 20-30% more than generic HDFS because it
> is implemented by single thread and executed by 1 client node.
> So the encryption can get more performance.
>
> May there be any error in my test?
>
> I know there are several implementation for encryting files in HDFS.
> Are these implementations enough to secure HDFS?
>
> best regards,
>
> seonpark
>
> * Sorry for my bad english
>
>
--
Have a Nice Day!
Lohit
+
Seonyeong Bak 2013-02-26, 05:33
+
Lance Norskog 2013-02-28, 00:26
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB