Hi,

  I am getting the following error in ApplicationMaster.stderr when running
an application with around 25 container launches. How can I resolve this
issue?

JVMDUMP006I Processing dump event "systhrow", detail
"java/lang/OutOfMemoryError" - please wait.
JVMDUMP032I JVM requested Heap dump using
'/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/heapdump.20130114.044646.16631.0001.phd'
in response to an event
JVMDUMP010I Heap dump written to
/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/heapdump.20130114.044646.16631.0001.phd
JVMDUMP032I JVM requested Java dump using
'/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/javacore.20130114.044646.16631.0002.txt'
in response to an event
JVMDUMP010I Java dump written to
/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/javacore.20130114.044646.16631.0002.txt
JVMDUMP032I JVM requested Snap dump using
'/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/Snap.20130114.044646.16631.0003.trc'
in response to an event
JVMDUMP010I Snap dump written to
/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/Snap.20130114.044646.16631.0003.trc
JVMDUMP013I Processed dump event "systhrow", detail
"java/lang/OutOfMemoryError".
Exception in thread "Thread-7" java.lang.OutOfMemoryError
        at ApplicationMaster.readMessage(ApplicationMaster.java:241)
        at
ApplicationMaster$SectionLeaderRunnable.run(ApplicationMaster.java:825)
        at java.lang.Thread.run(Thread.java:736)
Thanks,
Kishore

Hi Arun,

  Thanks for the reply. I am not running a Map Reduce application, running
some distributed application. And I am using 2.0.0-alpha. Also, I have one
more query.

 I am seeing that from the time ApplicationMaster is sumitted by Client to
the ASM part of AM, it is taking around 7 seconds for AM to come up. Is
there a way to improve that time?
Thanks,
Kishore
On Tue, Jan 15, 2013 at 5:43 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> How many maps & reduces did your job have? Also, what release are you
> using? I'd recommend at least 2.0.2-alpha, though we should be able to
> release 2.0.3-alpha very soon.
>
> Arun
>
> On Jan 14, 2013, at 4:35 AM, Krishna Kishore Bonagiri wrote:
>
> Hi,
>
>   I am getting the following error in ApplicationMaster.stderr when
> running an application with around 25 container launches. How can I resolve
> this issue?
>
> JVMDUMP006I Processing dump event "systhrow", detail
> "java/lang/OutOfMemoryError" - please wait.
> JVMDUMP032I JVM requested Heap dump using
> '/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/heapdump.20130114.044646.16631.0001.phd'
> in response to an event
> JVMDUMP010I Heap dump written to
> /tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/heapdump.20130114.044646.16631.0001.phd
> JVMDUMP032I JVM requested Java dump using
> '/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/javacore.20130114.044646.16631.0002.txt'
> in response to an event
> JVMDUMP010I Java dump written to
> /tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/javacore.20130114.044646.16631.0002.txt
> JVMDUMP032I JVM requested Snap dump using
> '/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/Snap.20130114.044646.16631.0003.trc'
> in response to an event
> JVMDUMP010I Snap dump written to
> /tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/Snap.20130114.044646.16631.0003.trc
> JVMDUMP013I Processed dump event "systhrow", detail
> "java/lang/OutOfMemoryError".
> Exception in thread "Thread-7" java.lang.OutOfMemoryError
>         at ApplicationMaster.readMessage(ApplicationMaster.java:241)
>         at
> ApplicationMaster$SectionLeaderRunnable.run(ApplicationMaster.java:825)
>         at java.lang.Thread.run(Thread.java:736)
>
>
> Thanks,
> Kishore
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>

Hi Anil,

  Thanks or the reply. I was trying google to know how to increase heap
size, and found that the option -Xmx1500m has be passed as command line
argument for java. Is that the way you are suggesting? If so, how can I
pass it for Application Master, because it is Client program that actually
launches the AM...
  Or is there any other way for doing it?

Thanks,
Kishore
On Tue, Jan 15, 2013 at 11:48 AM, anil gupta <[EMAIL PROTECTED]> wrote:

> The following log tells you the exact error:
>
> *JVMDUMP013I Processed dump event "systhrow", detail
> "java/lang/OutOfMemoryError".*
> * *
> *Exception in thread "Thread-7" java.lang.OutOfMemoryError*
> *        at ApplicationMaster.readMessage(**ApplicationMaster.java:241)*
> *        at ApplicationMaster$**SectionLeaderRunnable.run(**
> ApplicationMaster.java:825)*
> * *
> *        at java.lang.Thread.run(Thread.**java:736)*
>
> You might need to increase the HeapSize of ApplicationMaster.
>
> HTH,
> Anil Gupta
>
>
> On Mon, Jan 14, 2013 at 4:35 AM, Krishna Kishore Bonagiri <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>>   I am getting the following error in ApplicationMaster.stderr when
>> running an application with around 25 container launches. How can I resolve
>> this issue?
>>
>> JVMDUMP006I Processing dump event "systhrow", detail
>> "java/lang/OutOfMemoryError" - please wait.
>> JVMDUMP032I JVM requested Heap dump using
>> '/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/heapdump.20130114.044646.16631.0001.phd'
>> in response to an event
>> JVMDUMP010I Heap dump written to
>> /tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/heapdump.20130114.044646.16631.0001.phd
>> JVMDUMP032I JVM requested Java dump using
>> '/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/javacore.20130114.044646.16631.0002.txt'
>> in response to an event
>> JVMDUMP010I Java dump written to
>> /tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/javacore.20130114.044646.16631.0002.txt
>> JVMDUMP032I JVM requested Snap dump using
>> '/tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/Snap.20130114.044646.16631.0003.trc'
>> in response to an event
>> JVMDUMP010I Snap dump written to
>> /tmp/nm-local-dir/usercache/dsadm/appcache/application_1355219238448_0461/container_1355219238448_0461_01_000001/Snap.20130114.044646.16631.0003.trc
>> JVMDUMP013I Processed dump event "systhrow", detail
>> "java/lang/OutOfMemoryError".
>> Exception in thread "Thread-7" java.lang.OutOfMemoryError
>>         at ApplicationMaster.readMessage(ApplicationMaster.java:241)
>>         at
>> ApplicationMaster$SectionLeaderRunnable.run(ApplicationMaster.java:825)
>>         at java.lang.Thread.run(Thread.java:736)
>>
>>
>> Thanks,
>> Kishore
>>
>
>
>
> --
> Thanks & Regards,
> Anil Gupta