Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Put a file to HDFS with WebHDFS ?


Copy link to this message
-
Re: Put a file to HDFS with WebHDFS ?
The WebHDFS wiki has bad information regarding curl.  According to RFC 2616 Section 10.3.8,  a web client shall not automatically follow a 307 redirect unless the response is from a GET or HEAD request.   As jetty returns a 307 response for the PUT request against the WebHDFS api, curl does the proper thing and does not automatically follow the redirect.  You need to add "-L" to override the default behavior in curl, forcing it to follow the Location header in the HTTP response.   Here's an example from a secure grid.  

$ curl --negotiate -u : -i -X PUT -T /etc/motd -L 'http://mynamenode.example.com:50070/webhdfs/v1/tmp/motd?op=CREATE'
HTTP/1.1 100 Continue

HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=utf-8
WWW-Authenticate: Negotiate
Content-Length: 0
Server: Jetty(6.1.26)

HTTP/1.1 100 Continue

HTTP/1.1 307 TEMPORARY_REDIRECT
Content-Type: application/octet-stream
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Set-Cookie: hadoop.auth="xxxxx";Path=/
Location: http://datanode1.example.com:70/webhdfs/v1/tmp/motd?op=CREATE&delegation=xxxxx&overwrite=false
Content-Length: 0
Server: Jetty(6.1.26)

HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Content-Type: application/octet-stream
Location: webhdfs://mynamenode.example.com:50070/tmp/motd
Content-Length: 0
Server: Jetty(6.1.26)

-- Good luck, Adam

On Mar 19, 2013, at 11:19 PM, ken barclay <[EMAIL PROTECTED]> wrote:

> It didn't fail. If you hit the URL it returned, it will do the create.
>
> --- On Wed, 3/20/13, 小学园PHP <[EMAIL PROTECTED]> wrote:
>
> From: 小学园PHP <[EMAIL PROTECTED]>
> Subject: Put a file to HDFS with WebHDFS ?
> To: "user" <[EMAIL PROTECTED]>
> Date: Wednesday, March 20, 2013, 3:24 AM
>
> When i use the WebHDFS, i wanna put a file to Hadoop.
> my curl command:
> curl -i -X PUT -T /home/levi/3 "http://localhost:50070/webhdfs/v1/levi/3?op=CREATE"
>
> But i got a return:
> HTTP/1.1 100 Continue
>
> HTTP/1.1 307 TEMPORARY_REDIRECT
> Content-Type: application/octet-stream
> Location: http://58.53.211.46:50075/webhdfs/v1/levi/3?op=CREATE&overwrite=false
> Content-Length: 0
> Server: Jetty(6.1.26)
>
> OK, it failed.
> Do somebody know what has happened?

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB