Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: How to add another file system in Hadoop


+
Hemanth Yamijala 2013-02-21, 12:49
Copy link to this message
-
Re: How to add another file system in Hadoop
What Hemanth points to (fs.TYPE.impl, i.e. fs.cdmi.impl being set to
the classname of the custom FS impl.) is correct for the 1.x releases.
In 2.x and ahead, the class for a URI is auto-discovered from the
classpath (a 'service'). So as long as your jar is present on the
user's runtime, the FS class implementing cdmi:// will get loaded and
work without requiring a config injection globally like 1.x did.

On Thu, Feb 21, 2013 at 6:19 PM, Hemanth Yamijala
<[EMAIL PROTECTED]> wrote:
> I may be guessing here a bit. Basically a filesystem is identified by the
> protocol part of the URI of a file - so a file on the S3 filesystem will
> have a URI like s3://... If you look at the core-default.xml file in Hadoop
> source, you will see configuration keys like fs.<fs-type>.impl and the value
> is a class that implements that type of filesystem. So, you'd basically have
> to define a cdmi:// type protocol and define a fs.cdmi.impl key in your
> Hadoop configuration.
>
> Thanks
> hemanth
>
>
> On Thu, Feb 21, 2013 at 4:18 PM, Agarwal, Nikhil <[EMAIL PROTECTED]>
> wrote:
>>
>> Hi,
>>
>>
>>
>> I am planning to add a file system called CDMI under org.apache.hadoop.fs
>> in Hadoop, something similar to KFS or S3 which are already there under
>> org.apache.hadoop.fs. I wanted to ask that say, I write my file system for
>> CDMI and add the package under fs but then how do I tell the core-site.xml
>> or other configuration files to use CDMI file system. Where all do I need to
>> make changes to enable CDMI file system become a part of Hadoop ?
>>
>>
>>
>> Thanks a lot in advance.
>>
>>
>>
>> Regards,
>>
>> Nikhil
>
>

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB