Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Using REST to get ApplicationMaster info


Copy link to this message
-
Re: Using REST to get ApplicationMaster info (Issue solved)
Hmm, that is very odd.  It only checks the user if security is enabled to
warn the user about potentially accessing something unsafe.  I am not sure
why that would cause an issue.

--Bobby Evans

On 7/9/12 6:07 AM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:

>Hi Robert
>
>I figured out the problem just now. To avoid the below error, I had to set
>the 'hadoop.http.staticuser.user' property in core-site.xml (defaults to
>dr.who). I can now get runtime data from AppMaster using *curl* as well as
>in GUI.
>
>I wonder if we have to set this property even when we are not specifying
>the yarn web-proxy address (when it runs as part of RM by default) as
>well.
>If yes, was it documented somewhere which I failed to see? :(
>
>Anyways, thanks for your response so far.
>
>Regards,
>Prajakta
>
>
>
>On Mon, Jul 9, 2012 at 3:29 PM, Prajakta Kalmegh <[EMAIL PROTECTED]>
>wrote:
>
>> Hi Robert
>>
>> I started the proxyserver explicitly by specifying a value for the
>> yarn.web-proxy.address in yarn-site.xml. The proxyserver did start and I
>> tried getting the JSON response using the following command :
>>
>> curl --compressed -H "Accept: application/json" -X GET "
>>
>>http://localhost:8090/proxy/application_1341823967331_0001/ws/v1/mapreduc
>>e/jobs/job_1341823967331_0001
>> "
>>
>> However, it refused connection and below is the excerpt from the
>> Proxyserver logs:
>> ---------
>> 2012-07-09 14:26:40,402 INFO org.mortbay.log: Extract
>>
>>jar:file:/home/prajakta/Projects/IRL/hadoop-common/hadoop-dist/target/had
>>oop-3.0.0-SNAPSHOT/share/hadoop/mapreduce/hadoop-yarn-common-3.0.0-SNAPSH
>>OT.jar!/webapps/proxy
>> to /tmp/Jetty_localhost_8090_proxy____.ak3o30/webapp
>> 2012-07-09 14:26:40,992 INFO org.mortbay.log: Started
>> SelectChannelConnector@localhost:8090
>> 2012-07-09 14:26:40,993 INFO
>> org.apache.hadoop.yarn.service.AbstractService:
>> Service:org.apache.hadoop.yarn.server.webproxy.WebAppProxy is started.
>> 2012-07-09 14:26:40,993 INFO
>> org.apache.hadoop.yarn.service.AbstractService:
>> Service:org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer is
>>started.
>> 2012-07-09 14:33:26,039 INFO
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is
>> accessing unchecked
>> http://prajakta:44314/ws/v1/mapreduce/jobs/job_1341823967331_0001 which
>> is the app master GUI of application_1341823967331_0001 owned by
>>prajakta
>> 2012-07-09 14:33:29,277 INFO
>> org.apache.commons.httpclient.HttpMethodDirector: I/O exception
>> (org.apache.commons.httpclient.NoHttpResponseException) caught when
>> processing request: The server prajakta failed to respond
>> 2012-07-09 14:33:29,277 INFO
>> org.apache.commons.httpclient.HttpMethodDirector: Retrying request
>> 2012-07-09 14:33:29,284 WARN org.mortbay.log:
>>
>>/proxy/application_1341823967331_0001/ws/v1/mapreduce/jobs/job_1341823967
>>331_0001:
>> java.net.SocketException: Connection reset
>> 2012-07-09 14:37:33,834 INFO
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is
>> accessing unchecked
>>
>>http://prajakta:19888/jobhistory/job/job_1341823967331_0001/jobhistory/jo
>>b/job_1341823967331_0001which is the app master GUI of
>>application_1341823967331_0001 owned by
>> prajakta
>> ---------------
>>
>> I am not sure why http request object is setting my remoteUser to
>>dr.who.
>> :(
>>
>> I gather from <https://issues.apache.org/jira/browse/MAPREDUCE-2858>
>>that
>> this warning is posted only in case where security is disabled. I assume
>> that the proxy server is not disabled if security is disabled.
>>
>> Any idea what could be the reason for this I/O exception? Am I missing
>> setting any property for proper access. Please let me know.
>>
>> Regards,
>> Prajakta
>>
>>
>>
>>
>>
>>
>> On Fri, Jul 6, 2012 at 10:59 PM, Prajakta Kalmegh
>><[EMAIL PROTECTED]>wrote:
>>
>>> I am using hadoop trunk (forked from github). It supports RESTful APIs
>>>as
>>> I am able to retrieve JSON objects for RM (cluster/nodes info)+
>>> Historyserver. The only issue is with AppMaster REST API.