Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Need to add fs shim to use QFS


Copy link to this message
-
Re: Need to add fs shim to use QFS
Hi Steve,

Check out https://issues.apache.org/jira/browse/HADOOP-8886 for the
KFS removal. Seems relevant to your question here.

On Wed, Oct 10, 2012 at 12:52 AM, Steve Loughran <[EMAIL PROTECTED]> wrote:
> On 5 October 2012 18:27, Thilee Subramaniam <[EMAIL PROTECTED]> wrote:
>
>> We at Quantcast have released QFS 1.0 (Quantcast File System) to open
>> source. This is based on the KFS 0.5 (Kosmos Distributed File System),
>> a C++ distributed filesystem implementation. KFS plugs into Apache
>> Hadoop via the 'kfs' shim that is part of Hadoop codebase.
>>
>> QFS has added support for permissions, and also, provides fault tolerance
>> through Reed-Solomon encoding as well as replication. There are also a
>> number of performance and stability improvements, including a rewrite of
>> the client library to allow parallel concurrent I/Os. Going forward, new
>> releases of KFS will come from QFS.
>>
>>
> Does this mean the kfs plugin can go from the apache tree?
>
>
> One problem that we've always had with KFS is that nobody ever tested the
> filesystem, and it was inevitably out of sync with what was in KFS.
>
> Have you considered just pulling the kfs lib out and releasing the bridge
> classes yourself? It's what the other FS suppliers do, as it gives them
> more control over the libraries, including the ability to release more
> often.
>
> -steve

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB