Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Need to add fs shim to use QFS


+
Thilee Subramaniam 2012-10-05, 17:27
+
Steve Loughran 2012-10-09, 19:22
+
Harsh J 2012-10-10, 07:05
+
Thilee Subramaniam 2012-10-10, 15:03
+
Steve Loughran 2012-10-10, 19:14
+
Eli Collins 2012-10-10, 19:21
+
Steve Loughran 2012-10-11, 08:59
+
Thilee Subramaniam 2012-10-10, 23:34
Copy link to this message
-
Re: Need to add fs shim to use QFS
On 11 October 2012 00:34, Thilee Subramaniam <[EMAIL PROTECTED]> wrote:

>
>
> My initial goal was to make Hadoop use QFS the same way it used KFS. Since
> Hadoop branch-1 had lib/kfs.xx.jar, I was expecting to include a
> qfs.x.x.jar in the Hadoop release; my first patch was to use such jar. But
> now I see that Hadoop trunk links to external maven repos.
>
> It may be reasonable to link qfs.jar from an external source (I haven't
> yet figured out how to serve the maven repo from github for qfs.jar for
> Hadoop - any help on this will be appreciated).

The issue is not so much where qfs.jar comes from as where the
implementation of Hadoop FileSystem goes -the big question being: should it
live in your (OSS) codebase rather than Hadoop's? That would keep it 100%
in sync with your back end, give you the release schedule to suit you, and
ensure that bugreps end up in your bug reporting tools.
> This way your nightly
> builds will work and the tests can catch any qfs related regressions.
>

That regression testing is going to have be on QFS -which means your
infrastructure, real or virtual.
+
Thilee Subramaniam 2012-10-26, 00:24
+
Steve Loughran 2012-10-26, 17:32
+
Eli Collins 2012-10-05, 17:32
+
Thilee Subramaniam 2012-10-05, 23:27
+
Thilee Subramaniam 2012-10-09, 17:24