-Re: Any samples of how to write a custom FileSystem
Alejandro Abdelnur 2012-02-01, 05:20
You could also look at HttpFSFilesystem in the hadoop-httpfs module, it is
quite simple and selfcontained.
On Tue, Jan 31, 2012 at 8:37 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> To write a custom filesystem, extend on the FileSystem class.
> Depending on the scheme it is supposed to serve, creating an entry
> "fs.<scheme>.impl" in core-site.xml, and then loading it via the
> FileSystem.get(URI, conf) API will auto load it for you, provided the
> URI you pass has the right scheme.
> So supposing I have a FS scheme "foo", I'd register it in core-site.xml as:
> And then with a URI object from a path that goes "foo:///mypath/", I'd
> do: FileSystem.get(URI, new Configuration()) to get a FooFileSystem
> Similarly, if you want to overload the local filesystem with your
> class, override the "fs.file.impl" config with your derivative class,
> and that'd be used in your configuration loaded programs in future.
> A good, not-so-complex impl. example to look at generally would be the
> On Wed, Feb 1, 2012 at 9:41 AM, Steve Lewis <[EMAIL PROTECTED]> wrote:
> > Specifically how do I register a Custom FileSystem - any sample code
> > --
> > Steven M. Lewis PhD
> > 4221 105th Ave NE
> > Kirkland, WA 98033
> > 206-384-1340 (cell)
> > Skype lordjoe_com
> Harsh J
> Customer Ops. Engineer
> Cloudera | http://tiny.cloudera.com/about