Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - (Repost) Using REST to get ApplicationMaster info


Copy link to this message
-
Re: (Repost) Using REST to get ApplicationMaster info
Robert Evans 2012-07-25, 20:08
I am sorry it has taken me so long to respond.  Work has been crazy :).

I really am at a loss right now why you are getting the connection refused
error. The error is happening between the RM and the AM.  The Dr who is
something you can ignore.  It is the default name that is given to a web
user when security is disabled.  You probably want to check the logs for
the AM to see if there is anything in there, but beyond that I am at a
loss.

Sorry,

Bobby Evans

On 7/9/12 4:59 AM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:

>Hi Robert
>
>I started the proxyserver explicitly by specifying a value for the
>yarn.web-proxy.address in yarn-site.xml. The proxyserver did start and I
>tried getting the JSON response using the following command :
>
>curl --compressed -H "Accept: application/json" -X GET "
>http://localhost:8090/proxy/application_1341823967331_0001/ws/v1/mapreduce
>/jobs/job_1341823967331_0001
>"
>
>However, it refused connection and below is the excerpt from the
>Proxyserver logs:
>---------
>2012-07-09 14:26:40,402 INFO org.mortbay.log: Extract
>jar:file:/home/prajakta/Projects/IRL/hadoop-common/hadoop-dist/target/hado
>op-3.0.0-SNAPSHOT/share/hadoop/mapreduce/hadoop-yarn-common-3.0.0-SNAPSHOT
>.jar!/webapps/proxy
>to /tmp/Jetty_localhost_8090_proxy____.ak3o30/webapp
>2012-07-09 14:26:40,992 INFO org.mortbay.log: Started
>SelectChannelConnector@localhost:8090
>2012-07-09 14:26:40,993 INFO
>org.apache.hadoop.yarn.service.AbstractService:
>Service:org.apache.hadoop.yarn.server.webproxy.WebAppProxy is started.
>2012-07-09 14:26:40,993 INFO
>org.apache.hadoop.yarn.service.AbstractService:
>Service:org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer is
>started.
>2012-07-09 14:33:26,039 INFO
>org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is
>accessing unchecked
>http://prajakta:44314/ws/v1/mapreduce/jobs/job_1341823967331_0001 which is
>the app master GUI of application_1341823967331_0001 owned by prajakta
>2012-07-09 14:33:29,277 INFO
>org.apache.commons.httpclient.HttpMethodDirector: I/O exception
>(org.apache.commons.httpclient.NoHttpResponseException) caught when
>processing request: The server prajakta failed to respond
>2012-07-09 14:33:29,277 INFO
>org.apache.commons.httpclient.HttpMethodDirector: Retrying request
>2012-07-09 14:33:29,284 WARN org.mortbay.log:
>/proxy/application_1341823967331_0001/ws/v1/mapreduce/jobs/job_13418239673
>31_0001:
>java.net.SocketException: Connection reset
>2012-07-09 14:37:33,834 INFO
>org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is
>accessing unchecked
>http://prajakta:19888/jobhistory/job/job_1341823967331_0001/jobhistory/job
>/job_1341823967331_0001which
>is the app master GUI of application_1341823967331_0001 owned by
>prajakta
>---------------
>
>I am not sure why http request object is setting my remoteUser to dr.who.
>:(
>
>I gather from <https://issues.apache.org/jira/browse/MAPREDUCE-2858> that
>this warning is posted only in case where security is disabled. I assume
>that the proxy server is not disabled if security is disabled.
>
>Any idea what could be the reason for this I/O exception? Am I missing
>setting any property for proper access. Please let me know.
>
>Regards,
>Prajakta
>
>
>
>
>
>
>On Fri, Jul 6, 2012 at 10:59 PM, Prajakta Kalmegh
><[EMAIL PROTECTED]>wrote:
>
>> I am using hadoop trunk (forked from github). It supports RESTful APIs
>>as
>> I am able to retrieve JSON objects for RM (cluster/nodes info)+
>> Historyserver. The only issue is with AppMaster REST API.
>>
>> Regards,
>> Prajakta
>>
>>
>>
>> On Fri, Jul 6, 2012 at 10:55 PM, Robert Evans <[EMAIL PROTECTED]>
>>wrote:
>>
>>> What version of hadoop are you using?  It could be that the version you
>>> have does not have the RESTful APIs in it yet, and the proxy is working
>>> just fine.
>>>
>>> --Bobby Evans
>>>
>>> On 7/6/12 12:06 PM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:
>>>
>>> >Robert , Thanks for the response. If I do not provide any explicit