Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Using REST to get ApplicationMaster info


Copy link to this message
-
Re: Using REST to get ApplicationMaster info (Issue solved)
Robert Evans 2012-07-26, 18:29
OK I think I understand it now.  You probably have ACLs enabled, but no
web filter on the RM to let you sign in as a given user.  As such the
default filter is making you be Dr. Who, or whomever else it is, but the
ACL check in the web service is rejecting Dr Who, because that is not the
correct user.  You will probably run into this issue again if anyone else
but you runs something.  You could fix this by either disabling the ACL
check, which makes a lot of since for a cluster without security, or you
could implement a servlet Filter for the RM that would let you sign on as
a given user.

--Bobby Evans
On 7/26/12 12:48 AM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:

>Hi Bobby
>
>Thanks for the reply. My REST calls are working fine since I set the
>'hadoop.http.staticuser.user' property to 'prajakta' instead of Dr.Who in
>core-site.xml . I didn't get time to figure out the reason behind it as I
>just moved on to further coding :)
>
>Thanks,
>Prajakta
>
>
>
>On Thu, Jul 26, 2012 at 1:40 AM, Robert Evans <[EMAIL PROTECTED]> wrote:
>
>> Hmm, that is very odd.  It only checks the user if security is enabled
>>to
>> warn the user about potentially accessing something unsafe.  I am not
>>sure
>> why that would cause an issue.
>>
>> --Bobby Evans
>>
>> On 7/9/12 6:07 AM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:
>>
>> >Hi Robert
>> >
>> >I figured out the problem just now. To avoid the below error, I had to
>>set
>> >the 'hadoop.http.staticuser.user' property in core-site.xml (defaults
>>to
>> >dr.who). I can now get runtime data from AppMaster using *curl* as
>>well as
>> >in GUI.
>> >
>> >I wonder if we have to set this property even when we are not
>>specifying
>> >the yarn web-proxy address (when it runs as part of RM by default) as
>> >well.
>> >If yes, was it documented somewhere which I failed to see? :(
>> >
>> >Anyways, thanks for your response so far.
>> >
>> >Regards,
>> >Prajakta
>> >
>> >
>> >
>> >On Mon, Jul 9, 2012 at 3:29 PM, Prajakta Kalmegh <[EMAIL PROTECTED]>
>> >wrote:
>> >
>> >> Hi Robert
>> >>
>> >> I started the proxyserver explicitly by specifying a value for the
>> >> yarn.web-proxy.address in yarn-site.xml. The proxyserver did start
>>and I
>> >> tried getting the JSON response using the following command :
>> >>
>> >> curl --compressed -H "Accept: application/json" -X GET "
>> >>
>> >>
>>
>>http://localhost:8090/proxy/application_1341823967331_0001/ws/v1/mapreduc
>> >>e/jobs/job_1341823967331_0001
>> >> "
>> >>
>> >> However, it refused connection and below is the excerpt from the
>> >> Proxyserver logs:
>> >> ---------
>> >> 2012-07-09 14:26:40,402 INFO org.mortbay.log: Extract
>> >>
>>
>>>>jar:file:/home/prajakta/Projects/IRL/hadoop-common/hadoop-dist/target/h
>>>>ad
>>
>>>>oop-3.0.0-SNAPSHOT/share/hadoop/mapreduce/hadoop-yarn-common-3.0.0-SNAP
>>>>SH
>> >>OT.jar!/webapps/proxy
>> >> to /tmp/Jetty_localhost_8090_proxy____.ak3o30/webapp
>> >> 2012-07-09 14:26:40,992 INFO org.mortbay.log: Started
>> >> SelectChannelConnector@localhost:8090
>> >> 2012-07-09 14:26:40,993 INFO
>> >> org.apache.hadoop.yarn.service.AbstractService:
>> >> Service:org.apache.hadoop.yarn.server.webproxy.WebAppProxy is
>>started.
>> >> 2012-07-09 14:26:40,993 INFO
>> >> org.apache.hadoop.yarn.service.AbstractService:
>> >> Service:org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer is
>> >>started.
>> >> 2012-07-09 14:33:26,039 INFO
>> >> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is
>> >> accessing unchecked
>> >> http://prajakta:44314/ws/v1/mapreduce/jobs/job_1341823967331_0001
>>which
>> >> is the app master GUI of application_1341823967331_0001 owned by
>> >>prajakta
>> >> 2012-07-09 14:33:29,277 INFO
>> >> org.apache.commons.httpclient.HttpMethodDirector: I/O exception
>> >> (org.apache.commons.httpclient.NoHttpResponseException) caught when
>> >> processing request: The server prajakta failed to respond
>> >> 2012-07-09 14:33:29,277 INFO
>> >> org.apache.commons.httpclient.HttpMethodDirector: Retrying request