We at Quantcast have released QFS 1.0 (Quantcast File System) to open
source. This is based on the KFS 0.5 (Kosmos Distributed File System),
a C++ distributed filesystem implementation. KFS plugs into Apache
Hadoop via the 'kfs' shim that is part of Hadoop codebase.
QFS has added support for permissions, and also, provides fault tolerance
through Reed-Solomon encoding as well as replication. There are also a
number of performance and stability improvements, including a rewrite of
the client library to allow parallel concurrent I/Os. Going forward, new
releases of KFS will come from QFS.
The open source release of QFS is at http://quantcast.github.com/qfs
QFS plugs into Apache Hadoop the same way KFS does. Currently, one would
apply the patches or JARs from the QFS source tree onto Apache Hadoop to
make Hadoop use QFS. The patch for Apache Hadoop 1.0.X can be found at
In order to make the integration seamless, we would like to add a 'qfs'
shim to Apache Hadoop so that the current active branches (1.0.X, 2.X.X,
0.23.X) of Apache Hadoop can use QFS.
Towards this, I've submited an ASF JIRA feature ticket (HADOOP-8885) under
hadoop-common project, and send a pull request with the QFS shim changes
I will subsequently submit pull requests to the other active Hadoop
If you have any question, I will be happy yo answer or provide more
details on QFS.
Steve Loughran 2012-10-09, 19:22
Harsh J 2012-10-10, 07:05
Thilee Subramaniam 2012-10-10, 15:03
Steve Loughran 2012-10-10, 19:14
Eli Collins 2012-10-10, 19:21
Steve Loughran 2012-10-11, 08:59
Thilee Subramaniam 2012-10-10, 23:34
Steve Loughran 2012-10-11, 15:01
Thilee Subramaniam 2012-10-26, 00:24
Steve Loughran 2012-10-26, 17:32
Eli Collins 2012-10-05, 17:32
Thilee Subramaniam 2012-10-05, 23:27
Thilee Subramaniam 2012-10-09, 17:24