Hello,

HDFS supports http read-only access to filesystem. Is it possible to use wget to download file using some url like
http://<namenode>:<web<http://%3cnamenode%3e:%3cweb> gui port>/..

Thanks
Jaydeep
DISCLAIMER
=========This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

Sure you can. A http download option is also provided in the DataNode
web-interface (def-port:50075).

Use the "streamFile" feature of the same. An example follows.

If I have a file called 'results' lying as /user/hadoop/results, I'll do:
wget http://hostname.top.dom:50075/streamFile?filename=/user/hadoop/results
-O results

This will get me the file data in the wget-local file 'results'

On Tue, Jun 15, 2010 at 7:12 PM, Jaydeep Ayachit
<[EMAIL PROTECTED]> wrote:
> Hello,
>
> HDFS supports http read-only access to filesystem. Is it possible to use wget to download file using some url like
> http://<namenode>:<web<http://%3cnamenode%3e:%3cweb> gui port>/..
>
> Thanks
> Jaydeep
>
>
> DISCLAIMER
> =========> This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
>

--
Harsh J
www.harshj.com

Thanks, data node may not be known. Is it possible to direct url to namenode and namenode handling streaming by fetching data from various data nodes?

Regards
Jaydeep

-----Original Message-----
From: Harsh J [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, June 15, 2010 9:38 PM
To: [EMAIL PROTECTED]
Subject: Re: Using wget to download file from HDFS

Sure you can. A http download option is also provided in the DataNode
web-interface (def-port:50075).

Use the "streamFile" feature of the same. An example follows.

If I have a file called 'results' lying as /user/hadoop/results, I'll do:
wget http://hostname.top.dom:50075/streamFile?filename=/user/hadoop/results
-O results

This will get me the file data in the wget-local file 'results'

On Tue, Jun 15, 2010 at 7:12 PM, Jaydeep Ayachit
<[EMAIL PROTECTED]> wrote:
> Hello,
>
> HDFS supports http read-only access to filesystem. Is it possible to use wget to download file using some url like
> http://<namenode>:<web<http://%3cnamenode%3e:%3cweb> gui port>/..
>
> Thanks
> Jaydeep
>
>
> DISCLAIMER
> =========> This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
>

--
Harsh J
www.harshj.com

DISCLAIMER
=========This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

On Tue, Jun 15, 2010 at 12:30 PM, Jaydeep Ayachit <
[EMAIL PROTECTED]> wrote:

> Thanks, data node may not be known. Is it possible to direct url to
> namenode and namenode handling streaming by fetching data from various data
> nodes?
>
> Regards
> Jaydeep
>
> -----Original Message-----
> From: Harsh J [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, June 15, 2010 9:38 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Using wget to download file from HDFS
>
> Sure you can. A http download option is also provided in the DataNode
> web-interface (def-port:50075).
>
> Use the "streamFile" feature of the same. An example follows.
>
> If I have a file called 'results' lying as /user/hadoop/results, I'll do:
> wget
> http://hostname.top.dom:50075/streamFile?filename=/user/hadoop/results
> -O<http://hostname.top.dom:50075/streamFile?filename=/user/hadoop/results%0A-O>results
>
> This will get me the file data in the wget-local file 'results'
>
> On Tue, Jun 15, 2010 at 7:12 PM, Jaydeep Ayachit
> <[EMAIL PROTECTED]> wrote:
> > Hello,
> >
> > HDFS supports http read-only access to filesystem. Is it possible to use
> wget to download file using some url like
> > http://<namenode>:<web<http://%3cnamenode%3e:%3cweb> gui port>/..
> >
> > Thanks
> > Jaydeep
> >
> >
> > DISCLAIMER
> > =========> > This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
> >
>
>
>
> --
> Harsh J
> www.harshj.com
>
> DISCLAIMER
> =========> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>
To accomplish something like this: You have to use the name node web
interface and extract the names of datanodes from the html, then follow the
above process.

:::Edward reaches in his bag of tricks:::

Or you can kick up a webserver with tomcat to serve HDFS.

http://www.edwardcapriolo.com/wiki/en/Tomcat_Hadoop

On Jun 15, 2010, at 9:30 AM, Jaydeep Ayachit wrote:

> Thanks, data node may not be known. Is it possible to direct url to  
> namenode and namenode handling streaming by fetching data from  
> various data nodes?

If you access the servlet on the NameNode, it will automatically  
redirect you to a data node that has some of the data on it. You  
certainly should not pick a random data node yourself.

Also note that in yahoo 0.20.104 or 0.22, you'll need a Kerberos  
ticket or delegation token to use the servlet.

-- Owen