Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> [VOTE] Merge fs-encryption branch to trunk

Copy link to this message
[VOTE] Merge fs-encryption branch to trunk
Hi all,

I'd like to call a vote to merge the fs-encryption branch to trunk.
Development of this feature has been ongoing since March on HDFS-6134 and
HADOOP-10150, totally approximately 50 commits.

The fs-encryption branch introduces support for transparent, end-to-end
encryption within an "encryption zone". Each file stored within an
encryption zone is automatically encrypted and decrypted with a unique key.
These per-file keys are encrypted with an encryption key only accessible by
the client, ensuring that only the client is able to decrypt sensitive
data. Furthermore, there is support for native, hardware-accelerated AES
encryption. For further details, please see the design doc on HDFS-6134.

In terms of merge readiness, we've posted some successful consolidated
patches to the JIRA for Jenkins runs. distcp and fs -cp support has also
recently been completed, allowing users to securely copy encrypted files
without first decrypting them. There is ongoing work to add support for
WebHDFS, HttpFS, and other alternative access methods. Stephen Chu has also
posted a test plan, and has already identified a few issues that have been

Design and development of this feature was also a cross-company effort with
many different contributors.

I'd like to thank Charles Lamb, Yi Liu, Uma Maheswara Rao G, Colin McCabe,
and Juan Yu for their code contributions and reviews. Alejandro Abdelnur
was also instrumental, doing a lot of the design work and as well as
writing most of the Hadoop Key Mangement Server (KMS). Finally, I'd like to
thank everyone who gave feedback on the JIRAs. This includes Owen, Sanjay,
Larry, Mike Y, ATM, Todd, Nicholas, and Andy, among others.

With that, here's my +1 to merge this to trunk.