Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Using REST to get ApplicationMaster info


+
Prajakta Kalmegh 2012-06-28, 12:43
+
Harsh J 2012-06-28, 14:46
+
Prajakta Kalmegh 2012-06-28, 15:12
+
Robert Evans 2012-06-29, 15:25
+
Prajakta Kalmegh 2012-07-04, 12:25
+
Prajakta Kalmegh 2012-07-09, 11:07
+
Robert Evans 2012-07-25, 20:10
+
Prajakta Kalmegh 2012-07-26, 05:48
+
Robert Evans 2012-07-26, 18:29
+
Prajakta Kalmegh 2012-07-27, 08:11
Copy link to this message
-
Re: Using REST to get ApplicationMaster info (Issue solved)
Robert Evans 2012-07-27, 16:08
Yes you are right.  If it is true by default we probably want to update
the documentation for the web services to indicate this.  Could you file a
JIRA for improving that documentation?

Thanks,

Bobby

On 7/27/12 3:11 AM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:

>:) Yes, you are right. The yarn.acl.enable property in yarn-default.xml is
>set true. If the property is true by default, then this makes it mandatory
>for users to either specify a value for hadoop.http.staticuser.user
>property explicitly or to change the acl's to false. Am I right to assume
>this?
>
>Regards,
>Prajakta
>
>
>
>On Thu, Jul 26, 2012 at 11:59 PM, Robert Evans <[EMAIL PROTECTED]>
>wrote:
>
>> OK I think I understand it now.  You probably have ACLs enabled, but no
>> web filter on the RM to let you sign in as a given user.  As such the
>> default filter is making you be Dr. Who, or whomever else it is, but the
>> ACL check in the web service is rejecting Dr Who, because that is not
>>the
>> correct user.  You will probably run into this issue again if anyone
>>else
>> but you runs something.  You could fix this by either disabling the ACL
>> check, which makes a lot of since for a cluster without security, or you
>> could implement a servlet Filter for the RM that would let you sign on
>>as
>> a given user.
>>
>> --Bobby Evans
>>
>>
>> On 7/26/12 12:48 AM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:
>>
>> >Hi Bobby
>> >
>> >Thanks for the reply. My REST calls are working fine since I set the
>> >'hadoop.http.staticuser.user' property to 'prajakta' instead of Dr.Who
>>in
>> >core-site.xml . I didn't get time to figure out the reason behind it
>>as I
>> >just moved on to further coding :)
>> >
>> >Thanks,
>> >Prajakta
>> >
>> >
>> >
>> >On Thu, Jul 26, 2012 at 1:40 AM, Robert Evans <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> Hmm, that is very odd.  It only checks the user if security is
>>enabled
>> >>to
>> >> warn the user about potentially accessing something unsafe.  I am not
>> >>sure
>> >> why that would cause an issue.
>> >>
>> >> --Bobby Evans
>> >>
>> >> On 7/9/12 6:07 AM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:
>> >>
>> >> >Hi Robert
>> >> >
>> >> >I figured out the problem just now. To avoid the below error, I had
>>to
>> >>set
>> >> >the 'hadoop.http.staticuser.user' property in core-site.xml
>>(defaults
>> >>to
>> >> >dr.who). I can now get runtime data from AppMaster using *curl* as
>> >>well as
>> >> >in GUI.
>> >> >
>> >> >I wonder if we have to set this property even when we are not
>> >>specifying
>> >> >the yarn web-proxy address (when it runs as part of RM by default)
>>as
>> >> >well.
>> >> >If yes, was it documented somewhere which I failed to see? :(
>> >> >
>> >> >Anyways, thanks for your response so far.
>> >> >
>> >> >Regards,
>> >> >Prajakta
>> >> >
>> >> >
>> >> >
>> >> >On Mon, Jul 9, 2012 at 3:29 PM, Prajakta Kalmegh
>><[EMAIL PROTECTED]>
>> >> >wrote:
>> >> >
>> >> >> Hi Robert
>> >> >>
>> >> >> I started the proxyserver explicitly by specifying a value for the
>> >> >> yarn.web-proxy.address in yarn-site.xml. The proxyserver did start
>> >>and I
>> >> >> tried getting the JSON response using the following command :
>> >> >>
>> >> >> curl --compressed -H "Accept: application/json" -X GET "
>> >> >>
>> >> >>
>> >>
>> >>
>>
>>http://localhost:8090/proxy/application_1341823967331_0001/ws/v1/mapreduc
>> >> >>e/jobs/job_1341823967331_0001
>> >> >> "
>> >> >>
>> >> >> However, it refused connection and below is the excerpt from the
>> >> >> Proxyserver logs:
>> >> >> ---------
>> >> >> 2012-07-09 14:26:40,402 INFO org.mortbay.log: Extract
>> >> >>
>> >>
>>
>>>>>>jar:file:/home/prajakta/Projects/IRL/hadoop-common/hadoop-dist/target
>>>>>>/h
>> >>>>ad
>> >>
>>
>>>>>>oop-3.0.0-SNAPSHOT/share/hadoop/mapreduce/hadoop-yarn-common-3.0.0-SN
>>>>>>AP
>> >>>>SH
>> >> >>OT.jar!/webapps/proxy
>> >> >> to /tmp/Jetty_localhost_8090_proxy____.ak3o30/webapp
>> >> >> 2012-07-09 14:26:40,992 INFO org.mortbay.log: Started
>> >> >> SelectChannelConnector@localhost:8090