Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> RE: Debugging YARN AM


Copy link to this message
-
Re: Debugging YARN AM
Update: It looks like I could add the flag at line 515 of
hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
(package org.apache.hadoop.yarn.applications.distributedshell).

I tried this:
    vargs.add("-Xdebug
-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000");

I recompiled the jar and launched it with
hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command
whoami

I added a logging statement to make sure the change took effect. It prints,
but it doesn't seem to halt like the MR example (because of the suspend=y
option) even after launching the debugger. Why might this be?

Thanks!
Curtis

On Tue, Jun 18, 2013 at 8:23 AM, Curtis Ullerich <[EMAIL PROTECTED]>wrote:

> Hi Devaraj,
>
> That's what I'm not sure how to do. I want to debug by connecting
> remotely, and I'm not sure how to configure that in the code. I haven't
> found anything telling in the docs/source. Can you point me in the right
> direction?
>
> Thanks,
> Curtis
>
>
> On Mon, Jun 17, 2013 at 11:38 PM, Devaraj k <[EMAIL PROTECTED]> wrote:
>
>>  Hi Curtis, ****
>>
>> ** **
>>
>>      "yarn.app.mapreduce.am.command-opts" configuration is specific to
>> MRAppMaster. It is not applicable for DistributedShell AM.  ****
>>
>> ** **
>>
>> If you want to dump out debug information then you can make use of the
>> debug option of DistributedShell application. If you want to debug by
>> connecting remotely, you need to update the DS application code accordingly.
>> ****
>>
>> ** **
>>
>> Thanks****
>>
>> Devaraj K****
>>
>> ** **
>>
>> *From:* Curtis Ullerich [mailto:[EMAIL PROTECTED]]
>> *Sent:* 18 June 2013 08:19
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Debugging YARN AM****
>>
>> ** **
>>
>> Hi all, ****
>>
>> ** **
>>
>> I can successfully debug the MapReduce ApplicationMaster in standalone
>> mode by launching the pi estimator example with this command:****
>>
>> ** **
>>
>> hadoop jar hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar pi
>> "-Dyarn.app.mapreduce.am.command-opts=-Xdebug
>> -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000" 10 10****
>>
>> ** **
>>
>> and then attaching a debugger to port 8000 using Eclipse. This doesn't
>> work with the DistributedShell example, presumably because it's not
>> configurable with yarn.app.mapreduce.am.command as it's not MapReduce.
>> Looking in yarn-default.xml, I don't see an equivalent parameter. For
>> learning purposes, how can I debug the DistributedShell example (and other
>> AMs)?****
>>
>> ** **
>>
>> Thanks!****
>>
>> ** **
>>
>> Curtis****
>>
>
>