Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> How to add another file system in Hadoop


+
Agarwal, Nikhil 2013-02-21, 10:48
Copy link to this message
-
Re: How to add another file system in Hadoop
Hi  Agarwal,
   This repository and the corresponding README file may give you some hint
for the configuration.

   https://github.com/gluster/hadoop-glusterfs

yours,
Kun Ling

On Thu, Feb 21, 2013 at 8:53 PM, Harsh J <[EMAIL PROTECTED]> wrote:

> What Hemanth points to (fs.TYPE.impl, i.e. fs.cdmi.impl being set to
> the classname of the custom FS impl.) is correct for the 1.x releases.
> In 2.x and ahead, the class for a URI is auto-discovered from the
> classpath (a 'service'). So as long as your jar is present on the
> user's runtime, the FS class implementing cdmi:// will get loaded and
> work without requiring a config injection globally like 1.x did.
>
> On Thu, Feb 21, 2013 at 6:19 PM, Hemanth Yamijala
> <[EMAIL PROTECTED]> wrote:
> > I may be guessing here a bit. Basically a filesystem is identified by the
> > protocol part of the URI of a file - so a file on the S3 filesystem will
> > have a URI like s3://... If you look at the core-default.xml file in
> Hadoop
> > source, you will see configuration keys like fs.<fs-type>.impl and the
> value
> > is a class that implements that type of filesystem. So, you'd basically
> have
> > to define a cdmi:// type protocol and define a fs.cdmi.impl key in your
> > Hadoop configuration.
> >
> > Thanks
> > hemanth
> >
> >
> > On Thu, Feb 21, 2013 at 4:18 PM, Agarwal, Nikhil <
> [EMAIL PROTECTED]>
> > wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> I am planning to add a file system called CDMI under
> org.apache.hadoop.fs
> >> in Hadoop, something similar to KFS or S3 which are already there under
> >> org.apache.hadoop.fs. I wanted to ask that say, I write my file system
> for
> >> CDMI and add the package under fs but then how do I tell the
> core-site.xml
> >> or other configuration files to use CDMI file system. Where all do I
> need to
> >> make changes to enable CDMI file system become a part of Hadoop ?
> >>
> >>
> >>
> >> Thanks a lot in advance.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Nikhil
> >
> >
>
>
>
> --
> Harsh J
>

--
http://www.lingcc.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB