Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Using REST to get ApplicationMaster info


+
Prajakta Kalmegh 2012-06-28, 12:43
+
Harsh J 2012-06-28, 14:46
+
Prajakta Kalmegh 2012-06-28, 15:12
+
Robert Evans 2012-06-29, 15:25
+
Prajakta Kalmegh 2012-07-04, 12:25
+
Prajakta Kalmegh 2012-07-09, 11:07
Copy link to this message
-
Re: Using REST to get ApplicationMaster info (Issue solved)
Robert Evans 2012-07-25, 20:10
Hmm, that is very odd.  It only checks the user if security is enabled to
warn the user about potentially accessing something unsafe.  I am not sure
why that would cause an issue.

--Bobby Evans

On 7/9/12 6:07 AM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:

>Hi Robert
>
>I figured out the problem just now. To avoid the below error, I had to set
>the 'hadoop.http.staticuser.user' property in core-site.xml (defaults to
>dr.who). I can now get runtime data from AppMaster using *curl* as well as
>in GUI.
>
>I wonder if we have to set this property even when we are not specifying
>the yarn web-proxy address (when it runs as part of RM by default) as
>well.
>If yes, was it documented somewhere which I failed to see? :(
>
>Anyways, thanks for your response so far.
>
>Regards,
>Prajakta
>
>
>
>On Mon, Jul 9, 2012 at 3:29 PM, Prajakta Kalmegh <[EMAIL PROTECTED]>
>wrote:
>
>> Hi Robert
>>
>> I started the proxyserver explicitly by specifying a value for the
>> yarn.web-proxy.address in yarn-site.xml. The proxyserver did start and I
>> tried getting the JSON response using the following command :
>>
>> curl --compressed -H "Accept: application/json" -X GET "
>>
>>http://localhost:8090/proxy/application_1341823967331_0001/ws/v1/mapreduc
>>e/jobs/job_1341823967331_0001
>> "
>>
>> However, it refused connection and below is the excerpt from the
>> Proxyserver logs:
>> ---------
>> 2012-07-09 14:26:40,402 INFO org.mortbay.log: Extract
>>
>>jar:file:/home/prajakta/Projects/IRL/hadoop-common/hadoop-dist/target/had
>>oop-3.0.0-SNAPSHOT/share/hadoop/mapreduce/hadoop-yarn-common-3.0.0-SNAPSH
>>OT.jar!/webapps/proxy
>> to /tmp/Jetty_localhost_8090_proxy____.ak3o30/webapp
>> 2012-07-09 14:26:40,992 INFO org.mortbay.log: Started
>> SelectChannelConnector@localhost:8090
>> 2012-07-09 14:26:40,993 INFO
>> org.apache.hadoop.yarn.service.AbstractService:
>> Service:org.apache.hadoop.yarn.server.webproxy.WebAppProxy is started.
>> 2012-07-09 14:26:40,993 INFO
>> org.apache.hadoop.yarn.service.AbstractService:
>> Service:org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer is
>>started.
>> 2012-07-09 14:33:26,039 INFO
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is
>> accessing unchecked
>> http://prajakta:44314/ws/v1/mapreduce/jobs/job_1341823967331_0001 which
>> is the app master GUI of application_1341823967331_0001 owned by
>>prajakta
>> 2012-07-09 14:33:29,277 INFO
>> org.apache.commons.httpclient.HttpMethodDirector: I/O exception
>> (org.apache.commons.httpclient.NoHttpResponseException) caught when
>> processing request: The server prajakta failed to respond
>> 2012-07-09 14:33:29,277 INFO
>> org.apache.commons.httpclient.HttpMethodDirector: Retrying request
>> 2012-07-09 14:33:29,284 WARN org.mortbay.log:
>>
>>/proxy/application_1341823967331_0001/ws/v1/mapreduce/jobs/job_1341823967
>>331_0001:
>> java.net.SocketException: Connection reset
>> 2012-07-09 14:37:33,834 INFO
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is
>> accessing unchecked
>>
>>http://prajakta:19888/jobhistory/job/job_1341823967331_0001/jobhistory/jo
>>b/job_1341823967331_0001which is the app master GUI of
>>application_1341823967331_0001 owned by
>> prajakta
>> ---------------
>>
>> I am not sure why http request object is setting my remoteUser to
>>dr.who.
>> :(
>>
>> I gather from <https://issues.apache.org/jira/browse/MAPREDUCE-2858>
>>that
>> this warning is posted only in case where security is disabled. I assume
>> that the proxy server is not disabled if security is disabled.
>>
>> Any idea what could be the reason for this I/O exception? Am I missing
>> setting any property for proper access. Please let me know.
>>
>> Regards,
>> Prajakta
>>
>>
>>
>>
>>
>>
>> On Fri, Jul 6, 2012 at 10:59 PM, Prajakta Kalmegh
>><[EMAIL PROTECTED]>wrote:
>>
>>> I am using hadoop trunk (forked from github). It supports RESTful APIs
>>>as
>>> I am able to retrieve JSON objects for RM (cluster/nodes info)+
>>> Historyserver. The only issue is with AppMaster REST API.
+
Prajakta Kalmegh 2012-07-26, 05:48
+
Robert Evans 2012-07-26, 18:29
+
Prajakta Kalmegh 2012-07-27, 08:11
+
Robert Evans 2012-07-27, 16:08