Hey Guys,

I am wondering if the HDFS Client Libs will run under Windows. I saw
that in the sourcecode you use some shell commands to determine which
user is running the client etc.

Use-case is to have a Java App on Windows (XP, Server 2003 and upwards)
to read files via the Filesystem API.

If not, are there plans to implement that?

Thanks,
Tobi

Hey Tobi,
Some folks from the Condor team at Wisconsin claimed to have this working
(see https://issues.apache.org/jira/browse/HDFS-573), but never posted a
patch. Perhaps you could contact them and see if they have the patch ready
for public consumption?

Regards,
Jeff

On Thu, Oct 8, 2009 at 7:50 PM, Tobias N. Sasse <[EMAIL PROTECTED]> wrote:

> Hey Guys,
>
> I am wondering if the HDFS Client Libs will run under Windows. I saw that
> in the sourcecode you use some shell commands to determine which user is
> running the client etc.
>
> Use-case is to have a Java App on Windows (XP, Server 2003 and upwards) to
> read files via the Filesystem API.
>
> If not, are there plans to implement that?
>
> Thanks,
> Tobi
>

Thanks for that Info!

I have to work on the stable version, thus I need to know if this is
possible with the actual version (and if yes - which tweaks are necessary).

What I figured out is that you can specify the user/group login name
upfront within the conf, thus the hdfs client does not have to execute
any "whois" command or unix specific stuff.

So I guess if there is a patch it will be working with future releases.
But what about 0.18, 0.19 and 0.20?

Thanks for all further input on that,

Tobi
Jeff Hammerbacher wrote:
> Hey Tobi,
> Some folks from the Condor team at Wisconsin claimed to have this working
> (see https://issues.apache.org/jira/browse/HDFS-573), but never posted a
> patch. Perhaps you could contact them and see if they have the patch ready
> for public consumption?
>
> Regards,
> Jeff
>
> On Thu, Oct 8, 2009 at 7:50 PM, Tobias N. Sasse <[EMAIL PROTECTED]> wrote:
>
>  
>> Hey Guys,
>>
>> I am wondering if the HDFS Client Libs will run under Windows. I saw that
>> in the sourcecode you use some shell commands to determine which user is
>> running the client etc.
>>
>> Use-case is to have a Java App on Windows (XP, Server 2003 and upwards) to
>> read files via the Filesystem API.
>>
>> If not, are there plans to implement that?
>>
>> Thanks,
>> Tobi
>>
>>    
>
>

Hey Tobi,

Could you run the client using cygwin?

Thanks,
Eli
On Fri, Oct 9, 2009 at 12:30 AM, Tobias N. Sasse <[EMAIL PROTECTED]> wrote:
> Thanks for that Info!
>
> I have to work on the stable version, thus I need to know if this is
> possible with the actual version (and if yes - which tweaks are necessary).
>
> What I figured out is that you can specify the user/group login name upfront
> within the conf, thus the hdfs client does not have to execute any "whois"
> command or unix specific stuff.
>
> So I guess if there is a patch it will be working with future releases. But
> what about 0.18, 0.19 and 0.20?
>
> Thanks for all further input on that,
>
> Tobi
>
>
> Jeff Hammerbacher wrote:
>>
>> Hey Tobi,
>> Some folks from the Condor team at Wisconsin claimed to have this working
>> (see https://issues.apache.org/jira/browse/HDFS-573), but never posted a
>> patch. Perhaps you could contact them and see if they have the patch ready
>> for public consumption?
>>
>> Regards,
>> Jeff
>>
>> On Thu, Oct 8, 2009 at 7:50 PM, Tobias N. Sasse <[EMAIL PROTECTED]> wrote:
>>
>>
>>>
>>> Hey Guys,
>>>
>>> I am wondering if the HDFS Client Libs will run under Windows. I saw that
>>> in the sourcecode you use some shell commands to determine which user is
>>> running the client etc.
>>>
>>> Use-case is to have a Java App on Windows (XP, Server 2003 and upwards)
>>> to
>>> read files via the Filesystem API.
>>>
>>> If not, are there plans to implement that?
>>>
>>> Thanks,
>>> Tobi
>>>
>>>
>>
>>
>
>

Hey Eli,

no the requirement is plain native Windows :-/

Tobi

Eli Collins wrote:
> Hey Tobi,
>
> Could you run the client using cygwin?
>
> Thanks,
> Eli
>
>
> On Fri, Oct 9, 2009 at 12:30 AM, Tobias N. Sasse <[EMAIL PROTECTED]> wrote:
>  
>> Thanks for that Info!
>>
>> I have to work on the stable version, thus I need to know if this is
>> possible with the actual version (and if yes - which tweaks are necessary).
>>
>> What I figured out is that you can specify the user/group login name upfront
>> within the conf, thus the hdfs client does not have to execute any "whois"
>> command or unix specific stuff.
>>
>> So I guess if there is a patch it will be working with future releases. But
>> what about 0.18, 0.19 and 0.20?
>>
>> Thanks for all further input on that,
>>
>> Tobi
>>
>>
>> Jeff Hammerbacher wrote:
>>    
>>> Hey Tobi,
>>> Some folks from the Condor team at Wisconsin claimed to have this working
>>> (see https://issues.apache.org/jira/browse/HDFS-573), but never posted a
>>> patch. Perhaps you could contact them and see if they have the patch ready
>>> for public consumption?
>>>
>>> Regards,
>>> Jeff
>>>
>>> On Thu, Oct 8, 2009 at 7:50 PM, Tobias N. Sasse <[EMAIL PROTECTED]> wrote:
>>>
>>>
>>>      
>>>> Hey Guys,
>>>>
>>>> I am wondering if the HDFS Client Libs will run under Windows. I saw that
>>>> in the sourcecode you use some shell commands to determine which user is
>>>> running the client etc.
>>>>
>>>> Use-case is to have a Java App on Windows (XP, Server 2003 and upwards)
>>>> to
>>>> read files via the Filesystem API.
>>>>
>>>> If not, are there plans to implement that?
>>>>
>>>> Thanks,
>>>> Tobi
>>>>
>>>>
>>>>        
>>>      
>>    
>
>

cygwin is both plain and native to windows :)

I have the client working just fine with cygwin.

-- amr

Tobias N. Sasse wrote:
> Hey Eli,
>
> no the requirement is plain native Windows :-/
>
> Tobi
>
> Eli Collins wrote:
>> Hey Tobi,
>>
>> Could you run the client using cygwin?
>>
>> Thanks,
>> Eli
>>
>>
>> On Fri, Oct 9, 2009 at 12:30 AM, Tobias N. Sasse <[EMAIL PROTECTED]> wrote:
>>  
>>> Thanks for that Info!
>>>
>>> I have to work on the stable version, thus I need to know if this is
>>> possible with the actual version (and if yes - which tweaks are
>>> necessary).
>>>
>>> What I figured out is that you can specify the user/group login name
>>> upfront
>>> within the conf, thus the hdfs client does not have to execute any
>>> "whois"
>>> command or unix specific stuff.
>>>
>>> So I guess if there is a patch it will be working with future
>>> releases. But
>>> what about 0.18, 0.19 and 0.20?
>>>
>>> Thanks for all further input on that,
>>>
>>> Tobi
>>>
>>>
>>> Jeff Hammerbacher wrote:
>>>    
>>>> Hey Tobi,
>>>> Some folks from the Condor team at Wisconsin claimed to have this
>>>> working
>>>> (see https://issues.apache.org/jira/browse/HDFS-573), but never
>>>> posted a
>>>> patch. Perhaps you could contact them and see if they have the
>>>> patch ready
>>>> for public consumption?
>>>>
>>>> Regards,
>>>> Jeff
>>>>
>>>> On Thu, Oct 8, 2009 at 7:50 PM, Tobias N. Sasse <[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>>
>>>>      
>>>>> Hey Guys,
>>>>>
>>>>> I am wondering if the HDFS Client Libs will run under Windows. I
>>>>> saw that
>>>>> in the sourcecode you use some shell commands to determine which
>>>>> user is
>>>>> running the client etc.
>>>>>
>>>>> Use-case is to have a Java App on Windows (XP, Server 2003 and
>>>>> upwards)
>>>>> to
>>>>> read files via the Filesystem API.
>>>>>
>>>>> If not, are there plans to implement that?
>>>>>
>>>>> Thanks,
>>>>> Tobi
>>>>>
>>>>>
>>>>>        
>>>>      
>>>    
>>
>>  
>

Hi Amr,

thanks for the info, but cygwin is not an option.

Tobi

Amr Awadallah schrieb:
>
> cygwin is both plain and native to windows :)
>
> I have the client working just fine with cygwin.
>
> -- amr

another option is the "Cloudera Desktop", this way your windows users
can just us a browser to upload and download files.

http://cloudera.com/desktop

-- amr
> Hi Amr,
>
> thanks for the info, but cygwin is not an option.
>
> Tobi
>
> Amr Awadallah schrieb:
>>
>> cygwin is both plain and native to windows :)
>>
>> I have the client working just fine with cygwin.
>>
>> -- amr
>

Hi Amr,

I am planning to integrate the HDFS libraries into a program, which runs
on a variety of platforms. A major requirement is not to install any
tools, and to not use any form of virtualization or emulation. So the
base system is not touchable for me - I have to deal with what is there.
And that is only a JVM.

So my need is to have a statement if the HDFS Libs runs on Windows or
not, it's bad if not, but that's ok also - I am just looking for a
definite answer to this problem.

Thanks for the hint to Cloudera Desktop, but that is not solving my
problem sadly.

HTH,
Tobi

Amr Awadallah schrieb:
>
> another option is the "Cloudera Desktop", this way your windows users
> can just us a browser to upload and download files.
>
> http://cloudera.com/desktop
>
> -- amr

unfortunately the hadoop code base (both client and server) have a lot
of dependencies on unix-like commands, so the only way to make it work
would be to rewrite it to remove all such dependencies, which is a major
effort.

-- amr
> Hi Amr,
>
> I am planning to integrate the HDFS libraries into a program, which
> runs on a variety of platforms. A major requirement is not to install
> any tools, and to not use any form of virtualization or emulation. So
> the base system is not touchable for me - I have to deal with what is
> there. And that is only a JVM.
>
> So my need is to have a statement if the HDFS Libs runs on Windows or
> not, it's bad if not, but that's ok also - I am just looking for a
> definite answer to this problem.
>
> Thanks for the hint to Cloudera Desktop, but that is not solving my
> problem sadly.
>
> HTH,
> Tobi
>
> Amr Awadallah schrieb:
>>
>> another option is the "Cloudera Desktop", this way your windows users
>> can just us a browser to upload and download files.
>>
>> http://cloudera.com/desktop
>>
>> -- amr

On 10/11/09 2:26 PM, "Tobias N. Sasse" <[EMAIL PROTECTED]> wrote:
> I am planning to integrate the HDFS libraries into a program, which runs
> on a variety of platforms.

Be aware that HDFS (or MR for that matter) is not wire compatible with
itself across versions.