Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Need to add fs shim to use QFS


Copy link to this message
-
Re: Need to add fs shim to use QFS
On 5 October 2012 18:27, Thilee Subramaniam <[EMAIL PROTECTED]> wrote:

> We at Quantcast have released QFS 1.0 (Quantcast File System) to open
> source. This is based on the KFS 0.5 (Kosmos Distributed File System),
> a C++ distributed filesystem implementation. KFS plugs into Apache
> Hadoop via the 'kfs' shim that is part of Hadoop codebase.
>
> QFS has added support for permissions, and also, provides fault tolerance
> through Reed-Solomon encoding as well as replication. There are also a
> number of performance and stability improvements, including a rewrite of
> the client library to allow parallel concurrent I/Os. Going forward, new
> releases of KFS will come from QFS.
>
>
Does this mean the kfs plugin can go from the apache tree?
One problem that we've always had with KFS is that nobody ever tested the
filesystem, and it was inevitably out of sync with what was in KFS.

Have you considered just pulling the kfs lib out and releasing the bridge
classes yourself? It's what the other FS suppliers do, as it gives them
more control over the libraries, including the ability to release more
often.

-steve