Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Need to add fs shim to use QFS


+
Thilee Subramaniam 2012-10-05, 17:27
+
Steve Loughran 2012-10-09, 19:22
+
Harsh J 2012-10-10, 07:05
+
Thilee Subramaniam 2012-10-10, 15:03
+
Steve Loughran 2012-10-10, 19:14
+
Eli Collins 2012-10-10, 19:21
+
Steve Loughran 2012-10-11, 08:59
+
Thilee Subramaniam 2012-10-10, 23:34
Copy link to this message
-
Re: Need to add fs shim to use QFS
On 11 October 2012 00:34, Thilee Subramaniam <[EMAIL PROTECTED]> wrote:

>
>
> My initial goal was to make Hadoop use QFS the same way it used KFS. Since
> Hadoop branch-1 had lib/kfs.xx.jar, I was expecting to include a
> qfs.x.x.jar in the Hadoop release; my first patch was to use such jar. But
> now I see that Hadoop trunk links to external maven repos.
>
> It may be reasonable to link qfs.jar from an external source (I haven't
> yet figured out how to serve the maven repo from github for qfs.jar for
> Hadoop - any help on this will be appreciated).

The issue is not so much where qfs.jar comes from as where the
implementation of Hadoop FileSystem goes -the big question being: should it
live in your (OSS) codebase rather than Hadoop's? That would keep it 100%
in sync with your back end, give you the release schedule to suit you, and
ensure that bugreps end up in your bug reporting tools.
> This way your nightly
> builds will work and the tests can catch any qfs related regressions.
>

That regression testing is going to have be on QFS -which means your
infrastructure, real or virtual.
+
Thilee Subramaniam 2012-10-26, 00:24
+
Steve Loughran 2012-10-26, 17:32
+
Eli Collins 2012-10-05, 17:32
+
Thilee Subramaniam 2012-10-05, 23:27
+
Thilee Subramaniam 2012-10-09, 17:24
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB