Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Need to add fs shim to use QFS


+
Thilee Subramaniam 2012-10-05, 17:27
+
Steve Loughran 2012-10-09, 19:22
Copy link to this message
-
Re: Need to add fs shim to use QFS
Hi Steve,

Check out https://issues.apache.org/jira/browse/HADOOP-8886 for the
KFS removal. Seems relevant to your question here.

On Wed, Oct 10, 2012 at 12:52 AM, Steve Loughran <[EMAIL PROTECTED]> wrote:
> On 5 October 2012 18:27, Thilee Subramaniam <[EMAIL PROTECTED]> wrote:
>
>> We at Quantcast have released QFS 1.0 (Quantcast File System) to open
>> source. This is based on the KFS 0.5 (Kosmos Distributed File System),
>> a C++ distributed filesystem implementation. KFS plugs into Apache
>> Hadoop via the 'kfs' shim that is part of Hadoop codebase.
>>
>> QFS has added support for permissions, and also, provides fault tolerance
>> through Reed-Solomon encoding as well as replication. There are also a
>> number of performance and stability improvements, including a rewrite of
>> the client library to allow parallel concurrent I/Os. Going forward, new
>> releases of KFS will come from QFS.
>>
>>
> Does this mean the kfs plugin can go from the apache tree?
>
>
> One problem that we've always had with KFS is that nobody ever tested the
> filesystem, and it was inevitably out of sync with what was in KFS.
>
> Have you considered just pulling the kfs lib out and releasing the bridge
> classes yourself? It's what the other FS suppliers do, as it gives them
> more control over the libraries, including the ability to release more
> often.
>
> -steve

--
Harsh J
+
Thilee Subramaniam 2012-10-10, 15:03
+
Steve Loughran 2012-10-10, 19:14
+
Eli Collins 2012-10-10, 19:21
+
Steve Loughran 2012-10-11, 08:59
+
Thilee Subramaniam 2012-10-10, 23:34
+
Steve Loughran 2012-10-11, 15:01
+
Thilee Subramaniam 2012-10-26, 00:24
+
Steve Loughran 2012-10-26, 17:32
+
Eli Collins 2012-10-05, 17:32
+
Thilee Subramaniam 2012-10-05, 23:27
+
Thilee Subramaniam 2012-10-09, 17:24