Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Need to add fs shim to use QFS


+
Thilee Subramaniam 2012-10-05, 17:27
+
Steve Loughran 2012-10-09, 19:22
+
Harsh J 2012-10-10, 07:05
+
Thilee Subramaniam 2012-10-10, 15:03
+
Steve Loughran 2012-10-10, 19:14
+
Eli Collins 2012-10-10, 19:21
+
Steve Loughran 2012-10-11, 08:59
+
Thilee Subramaniam 2012-10-10, 23:34
+
Steve Loughran 2012-10-11, 15:01
+
Thilee Subramaniam 2012-10-26, 00:24
+
Steve Loughran 2012-10-26, 17:32
Copy link to this message
-
Re: Need to add fs shim to use QFS
Hey Thilee,

Thanks for contributing. We don't process pull request on the git
mirrors, please upload a patch against trunk and branch-1 if you'd
like this included in Hadoop 1.x and 2.x releases. More info here:
http://wiki.apache.org/hadoop/HowToContribute

Thanks,
Eli

On Fri, Oct 5, 2012 at 10:27 AM, Thilee Subramaniam
<[EMAIL PROTECTED]> wrote:
> We at Quantcast have released QFS 1.0 (Quantcast File System) to open
> source. This is based on the KFS 0.5 (Kosmos Distributed File System),
> a C++ distributed filesystem implementation. KFS plugs into Apache
> Hadoop via the 'kfs' shim that is part of Hadoop codebase.
>
> QFS has added support for permissions, and also, provides fault tolerance
> through Reed-Solomon encoding as well as replication. There are also a
> number of performance and stability improvements, including a rewrite of
> the client library to allow parallel concurrent I/Os. Going forward, new
> releases of KFS will come from QFS.
>
> The open source release of QFS is at http://quantcast.github.com/qfs
>
> QFS plugs into Apache Hadoop the same way KFS does. Currently, one would
> apply the patches or JARs from the QFS source tree onto Apache Hadoop to
> make Hadoop use QFS. The patch for Apache Hadoop 1.0.X can be found at
> https://github.com/quantcast/qfs/blob/master/hadoop/hadoop-1.0.X.patch
>
> In order to make the integration seamless, we would like to add a 'qfs'
> shim to Apache Hadoop so that the current active branches (1.0.X, 2.X.X,
> 0.23.X) of Apache Hadoop can use QFS.
>
> Towards this, I've submited an ASF JIRA feature ticket (HADOOP-8885) under
> hadoop-common project, and send a pull request with the QFS shim changes
> to https://github.com/apache/hadoop-common/tree/branch-1.0.2
>
> I will subsequently submit pull requests to the other active Hadoop
> branches.
>
> If you have any question, I will be happy yo answer or provide more
> details on QFS.
>
> - Thilee
>
+
Thilee Subramaniam 2012-10-05, 23:27
+
Thilee Subramaniam 2012-10-09, 17:24
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB