Hi Folks,
This might be a stupid question, but I'm new to Java and Hadoop, so...

Anyway, if I want to check what FileSystem is currently being used at
some point (i.e. evaluating FileSystem.get(conf)), what would be the
most elegant way of doing that? Should I just do something like:
     if (FileSystem.get(conf) == "HDFS") {...}
Or is there a better way?

Eli

On Tue, Feb 7, 2012 at 5:24 PM, Eli Finkelshteyn <[EMAIL PROTECTED]> wrote:

> Hi Folks,
> This might be a stupid question, but I'm new to Java and Hadoop, so...
>
> Anyway, if I want to check what FileSystem is currently being used at some
> point (i.e. evaluating FileSystem.get(conf)), what would be the most
> elegant way of doing that? Should I just do something like:
>    if (FileSystem.get(conf) == "HDFS") {...}
> Or is there a better way?
>
> Eli
>

conf.get("fs.default.name") would return a URI such as hdfs://bla:8000 or
file:///this. Although an application could have two Configurations which
could be used to connect to two separate FileSystems inside the same java
application.

Edward

Thanks! I wound up using fs.getUri(), so I can do stuff like
fs.getUri().toString().matches("^hdfs://.*").

On 2/7/12 5:42 PM, Edward Capriolo wrote:
> On Tue, Feb 7, 2012 at 5:24 PM, Eli Finkelshteyn<[EMAIL PROTECTED]>  wrote:
>
>> Hi Folks,
>> This might be a stupid question, but I'm new to Java and Hadoop, so...
>>
>> Anyway, if I want to check what FileSystem is currently being used at some
>> point (i.e. evaluating FileSystem.get(conf)), what would be the most
>> elegant way of doing that? Should I just do something like:
>>     if (FileSystem.get(conf) == "HDFS") {...}
>> Or is there a better way?
>>
>> Eli
>>
> conf.get("fs.default.name") would return a URI such as hdfs://bla:8000 or
> file:///this. Although an application could have two Configurations which
> could be used to connect to two separate FileSystems inside the same java
> application.
>
> Edward
>

The right way to get a specific filesystem would be to load it with
the URI itself -- FileSystem.get(URI, conf) for example, and that
wouldn't go wrong.

Just curious on what you're trying to solve here, that you'd need to
check the instance?

On Wed, Feb 8, 2012 at 4:19 AM, Eli Finkelshteyn <[EMAIL PROTECTED]> wrote:
> Thanks! I wound up using fs.getUri(), so I can do stuff like
> fs.getUri().toString().matches("^hdfs://.*").
>
>
> On 2/7/12 5:42 PM, Edward Capriolo wrote:
>>
>> On Tue, Feb 7, 2012 at 5:24 PM, Eli Finkelshteyn<[EMAIL PROTECTED]>
>>  wrote:
>>
>>> Hi Folks,
>>> This might be a stupid question, but I'm new to Java and Hadoop, so...
>>>
>>> Anyway, if I want to check what FileSystem is currently being used at
>>> some
>>> point (i.e. evaluating FileSystem.get(conf)), what would be the most
>>> elegant way of doing that? Should I just do something like:
>>>    if (FileSystem.get(conf) == "HDFS") {...}
>>> Or is there a better way?
>>>
>>> Eli
>>>
>> conf.get("fs.default.name") would return a URI such as hdfs://bla:8000 or
>> file:///this. Although an application could have two Configurations which
>> could be used to connect to two separate FileSystems inside the same java
>> application.
>>
>> Edward
>>
>

--
Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about

It's kind of silly and unnecessary, but I have a task where I needed to
take an implementation of a recommender in stand-alone Mahout and get it
to optionally pull files from HDFS. Since nothing is really built to
support this because if you're using HDFS it's assumed that you'd be
using Mahout over Hadoop, and because the person who built the
recommender used local File objects all over the place, which you can't
really convert to from Hadoop Path objects (and rightfully so), I had to
hack at things quite a bit.

Anyway, the upshot is that at a couple of points throughout the program,
I needed to figure out what file system was being used, so I could
appropriately handle whether I could use File objects directly on a
given URI or not.

Eli

On 2/7/12 10:47 PM, Harsh J wrote:
> The right way to get a specific filesystem would be to load it with
> the URI itself -- FileSystem.get(URI, conf) for example, and that
> wouldn't go wrong.
>
> Just curious on what you're trying to solve here, that you'd need to
> check the instance?
>
> On Wed, Feb 8, 2012 at 4:19 AM, Eli Finkelshteyn<[EMAIL PROTECTED]>  wrote:
>> Thanks! I wound up using fs.getUri(), so I can do stuff like
>> fs.getUri().toString().matches("^hdfs://.*").
>>
>>
>> On 2/7/12 5:42 PM, Edward Capriolo wrote:
>>> On Tue, Feb 7, 2012 at 5:24 PM, Eli Finkelshteyn<[EMAIL PROTECTED]>
>>>   wrote:
>>>
>>>> Hi Folks,
>>>> This might be a stupid question, but I'm new to Java and Hadoop, so...
>>>>
>>>> Anyway, if I want to check what FileSystem is currently being used at
>>>> some
>>>> point (i.e. evaluating FileSystem.get(conf)), what would be the most
>>>> elegant way of doing that? Should I just do something like:
>>>>     if (FileSystem.get(conf) == "HDFS") {...}
>>>> Or is there a better way?
>>>>
>>>> Eli
>>>>
>>> conf.get("fs.default.name") would return a URI such as hdfs://bla:8000 or
>>> file:///this. Although an application could have two Configurations which
>>> could be used to connect to two separate FileSystems inside the same java
>>> application.
>>>
>>> Edward
>>>
>
>