Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> RE: Debugging YARN AM


Copy link to this message
-
Re: Debugging YARN AM
Update: It looks like I could add the flag at line 515 of
hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
(package org.apache.hadoop.yarn.applications.distributedshell).

I tried this:
    vargs.add("-Xdebug
-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000");

I recompiled the jar and launched it with
hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command
whoami

I added a logging statement to make sure the change took effect. It prints,
but it doesn't seem to halt like the MR example (because of the suspend=y
option) even after launching the debugger. Why might this be?

Thanks!
Curtis

On Tue, Jun 18, 2013 at 8:23 AM, Curtis Ullerich <[EMAIL PROTECTED]>wrote:

> Hi Devaraj,
>
> That's what I'm not sure how to do. I want to debug by connecting
> remotely, and I'm not sure how to configure that in the code. I haven't
> found anything telling in the docs/source. Can you point me in the right
> direction?
>
> Thanks,
> Curtis
>
>
> On Mon, Jun 17, 2013 at 11:38 PM, Devaraj k <[EMAIL PROTECTED]> wrote:
>
>>  Hi Curtis, ****
>>
>> ** **
>>
>>      "yarn.app.mapreduce.am.command-opts" configuration is specific to
>> MRAppMaster. It is not applicable for DistributedShell AM.  ****
>>
>> ** **
>>
>> If you want to dump out debug information then you can make use of the
>> debug option of DistributedShell application. If you want to debug by
>> connecting remotely, you need to update the DS application code accordingly.
>> ****
>>
>> ** **
>>
>> Thanks****
>>
>> Devaraj K****
>>
>> ** **
>>
>> *From:* Curtis Ullerich [mailto:[EMAIL PROTECTED]]
>> *Sent:* 18 June 2013 08:19
>> *To:* [EMAIL PROTECTED]
>> *Subject:* Debugging YARN AM****
>>
>> ** **
>>
>> Hi all, ****
>>
>> ** **
>>
>> I can successfully debug the MapReduce ApplicationMaster in standalone
>> mode by launching the pi estimator example with this command:****
>>
>> ** **
>>
>> hadoop jar hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar pi
>> "-Dyarn.app.mapreduce.am.command-opts=-Xdebug
>> -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000" 10 10****
>>
>> ** **
>>
>> and then attaching a debugger to port 8000 using Eclipse. This doesn't
>> work with the DistributedShell example, presumably because it's not
>> configurable with yarn.app.mapreduce.am.command as it's not MapReduce.
>> Looking in yarn-default.xml, I don't see an equivalent parameter. For
>> learning purposes, how can I debug the DistributedShell example (and other
>> AMs)?****
>>
>> ** **
>>
>> Thanks!****
>>
>> ** **
>>
>> Curtis****
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB