Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> (Repost) Using REST to get ApplicationMaster info


Copy link to this message
-
Re: (Repost) Using REST to get ApplicationMaster info
What version of hadoop are you using?  It could be that the version you
have does not have the RESTful APIs in it yet, and the proxy is working
just fine.

--Bobby Evans

On 7/6/12 12:06 PM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:

>Robert , Thanks for the response. If I do not provide any explicit
>configuration for the proxy server, do I still need to start it using the
>'yarn start proxy server'? I am currently not doing it.
>
>Also, I am able to access the html page for proxy using the
><http://localhost:8088/proxy/{appid}/mapreduce/jobs> URL. (Note this url
>does not have the '/ws/v1/ part in it. I get the html response when I
>query
>for this URL in runtime.
>
>So I assume the proxy server must be starting fine since I am able to
>access this URL. I will try logging more details tomorrow from my office
>machine and will let you know the result.
>
>Regards,
>Prajakta
>
>
>
>On Fri, Jul 6, 2012 at 10:22 PM, Robert Evans <[EMAIL PROTECTED]> wrote:
>
>> Sorry I did not respond sooner.  The default behavior is to have the
>>proxy
>> server run as part of the RM.  I am not really sure why it is not doing
>> this in your case.  If you set the config yourself to be a URI that is
>> different from that of the RM then you need to launch a standalone proxy
>> server.  You can do this by running
>>
>> yarn start proxy server
>>
>> Without sitting down with you it is going to be somewhat difficult to
>> debug why this is happening.  However, in retrospect it would be nice to
>> add in some extra logging to help indicate why the proxy server is not
>> functioning as desired.  If you could file a JIRA to add in the logging
>>I
>> would be happy to provide a patch to you and we can try and debug the
>> issue further.  Please file it under the MAPREDUCE JIRA project.
>>
>> --Bobby
>>
>> On 7/6/12 3:29 AM, "Prajakta Kalmegh" <[EMAIL PROTECTED]> wrote:
>>
>> >Re-posting as I haven't got a solution yet. Sorry for spamming. I
>>won't be
>> >able to proceed in my code until I get a JSON response using AppMaster
>> >REST
>> >URL. :(
>> >
>> >Thanks,
>> >Prajakta
>> >
>> >
>> >On Wed, Jul 4, 2012 at 5:55 PM, Prajakta Kalmegh <[EMAIL PROTECTED]>
>> >wrote:
>> >
>> >> Hi Robert/Harsh
>> >>
>> >> Thanks for your reply.
>> >>
>> >> My RM is starting just fine. The problem is with the use of
>> >>http://<proxy httpddress:port>/proxy/{appid}/ws/v1/mapreduce
>> >> to get the JSON response.
>> >>
>> >> As I said before, I had not configured the yarn.web-proxy.address
>> >>property in yarn-site.xml. I assumed it will use the RM's
>> >>yarn.resourcemanager.webapp.address property value as default.
>>However,
>> >>it gives me a '404-Page not found error'.  Today I tried specifying a
>> >>value explicitly for the yarn.web-proxy.address property.
>> >>
>> >> On running the wordcount example, it even gives a url
>> >><http://localhost:8090>/proxy/{appid}/> to track the App Mast info.
>> >>However, I am still not able to get a json response.
>> >>
>> >> Also, I tried to get the data from historyserver instead of runtime
>> >>using the instructions given on page
>> >><
>> http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yar
>> >>n-site/HistoryServerRest.html>
>> >>
>> >> HistoryServer REST response does not give me jobids corresponding to
>>an
>> >>application. It just lists all the jobs run until now. By the way, the
>> >>documentation does say
>> >>
>> >> ----------
>> >>
>> >> "Both of the following URI's give you the history server information,
>> >>from an application id identified by the appid value.
>> >>   * http://<history server http address:port>/ws/v1/history
>> >>   * http://<history server http address:port>/ws/v1/history/info"
>> >> ---------
>> >>
>> >> But there is no provision to specify the application id with these
>>REST
>> >>URLs.
>> >>
>> >> Any idea how I can get the Application Master REST working and also
>> >>linking jobids to application id using the HistoryServerREST API?
>> >>
>> >> Any help is appreciated. Thanks in advance.