Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Put a file to HDFS with WebHDFS ?


Copy link to this message
-
Re: Put a file to HDFS with WebHDFS ?
The WebHDFS wiki has bad information regarding curl.  According to RFC 2616 Section 10.3.8,  a web client shall not automatically follow a 307 redirect unless the response is from a GET or HEAD request.   As jetty returns a 307 response for the PUT request against the WebHDFS api, curl does the proper thing and does not automatically follow the redirect.  You need to add "-L" to override the default behavior in curl, forcing it to follow the Location header in the HTTP response.   Here's an example from a secure grid.  

$ curl --negotiate -u : -i -X PUT -T /etc/motd -L 'http://mynamenode.example.com:50070/webhdfs/v1/tmp/motd?op=CREATE'
HTTP/1.1 100 Continue

HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=utf-8
WWW-Authenticate: Negotiate
Content-Length: 0
Server: Jetty(6.1.26)

HTTP/1.1 100 Continue

HTTP/1.1 307 TEMPORARY_REDIRECT
Content-Type: application/octet-stream
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Set-Cookie: hadoop.auth="xxxxx";Path=/
Location: http://datanode1.example.com:70/webhdfs/v1/tmp/motd?op=CREATE&delegation=xxxxx&overwrite=false
Content-Length: 0
Server: Jetty(6.1.26)

HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Content-Type: application/octet-stream
Location: webhdfs://mynamenode.example.com:50070/tmp/motd
Content-Length: 0
Server: Jetty(6.1.26)

-- Good luck, Adam

On Mar 19, 2013, at 11:19 PM, ken barclay <[EMAIL PROTECTED]> wrote:

> It didn't fail. If you hit the URL it returned, it will do the create.
>
> --- On Wed, 3/20/13, 小学园PHP <[EMAIL PROTECTED]> wrote:
>
> From: 小学园PHP <[EMAIL PROTECTED]>
> Subject: Put a file to HDFS with WebHDFS ?
> To: "user" <[EMAIL PROTECTED]>
> Date: Wednesday, March 20, 2013, 3:24 AM
>
> When i use the WebHDFS, i wanna put a file to Hadoop.
> my curl command:
> curl -i -X PUT -T /home/levi/3 "http://localhost:50070/webhdfs/v1/levi/3?op=CREATE"
>
> But i got a return:
> HTTP/1.1 100 Continue
>
> HTTP/1.1 307 TEMPORARY_REDIRECT
> Content-Type: application/octet-stream
> Location: http://58.53.211.46:50075/webhdfs/v1/levi/3?op=CREATE&overwrite=false
> Content-Length: 0
> Server: Jetty(6.1.26)
>
> OK, it failed.
> Do somebody know what has happened?