Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Where should we host Hadoop FileSystem plugins for 3rd Party FileSystems?


Copy link to this message
-
Re: Where should we host Hadoop FileSystem plugins for 3rd Party FileSystems?
>From what I have seen, that is consistent with the modifications in the code.

E.g. - Removal of kfs, and qfs shim is now external on github.  

Cheers,
Tim

----- Original Message -----
> From: "Stephen Watt" <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Tuesday, May 21, 2013 11:49:34 AM
> Subject: Where should we host Hadoop FileSystem plugins for 3rd Party FileSystems?
>
> Hi Folks
>
> My name is Steve Watt and I am presently working on enabling glusterfs to be
> used as a Hadoop FileSystem. Most of the work thus far has involved
> developing a Hadoop FileSystem plugin for glusterfs. I'm getting to the
> point where the plugin is becoming stable and I've been trying to understand
> where the right place is to host/manage/version it.
>
> Steve Loughran was kind enough to point out a few past threads in the
> community
> (http://lucene.472066.n3.nabble.com/Need-to-add-fs-shim-to-use-QFS-td4012118.html)
> that show a disposition to move away from Hadoop Common containing client
> code (plugins) for 3rd party FileSystems. This makes sense and allows the
> filesystem plugin developer more autonomy as well as reduces Hadoop Common's
> dependence on 3rd Party libraries. I'm easy either way. I just wanted to
> verify that the community's preference is still to have client code for 3rd
> Party FileSystems hosted and managed outside of Hadoop Common before I take
> that direction.
>
> Regards
> Steve Watt
>