Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: Time taken for starting AMRMClientAsync


Copy link to this message
-
Re: Time taken for starting AMRMClientAsync
Krishna Kishore Bonagiri 2013-11-26, 02:40
Hi Alejandro,

  I don't start all the AMs from the same JVM. How can I do that? Also,
when I do that, that will save me time taken to get AM started, which is
also good to see an improvement in. Please let me know how can I do that?
And, would this also save me time taken for connecting from AM to the
Resource Manager?

Thanks,
Kishore
On Tue, Nov 26, 2013 at 3:45 AM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:

> Hi Krishna,
>
> Are you starting all AMs from the same JVM? Mind sharing the code you are
> using for your time testing?
>
> Thx
>
>
> On Thu, Nov 21, 2013 at 6:11 AM, Krishna Kishore Bonagiri <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Alejandro,
>>
>>  I have modified the code in
>>
>>
>> hadoop-2.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher/src/main/java/org/apache/hadoop/yarn/applications/unmanagedamlauncher/UnmanagedAMLauncher.java
>>
>> to submit multiple application masters one after another and still seeing
>> 800 to 900 ms being taken for the start() call on AMRMClientAsync in all
>> of those applications.
>>
>> Please suggest if you think I am missing something else
>>
>> Thanks,
>> Kishore
>>
>>
>> On Tue, Nov 19, 2013 at 6:07 PM, Krishna Kishore Bonagiri <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi Alejandro,
>>>
>>>   I don't know what are managed and unmanaged AMs, can you please
>>> explain me what are the difference and how are each of them launched?
>>>
>>>  I tried to google for these terms and came
>>> across hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar, is it
>>> related to that?
>>>
>>> Thanks,
>>> Kishore
>>>
>>>
>>> On Tue, Nov 19, 2013 at 12:15 AM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>>>
>>>> Kishore,
>>>>
>>>> Also, please specify if you are using managed or unmanaged AMs (the
>>>> numbers I've mentioned before are using unmanaged AMs).
>>>>
>>>> thx
>>>>
>>>>
>>>> On Sun, Nov 17, 2013 at 11:16 AM, Vinod Kumar Vavilapalli <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> It is just creating a connection to RM and shouldn't take that long.
>>>>> Can you please file a ticket so that we can look at it?
>>>>>
>>>>> JVM class loading overhead is one possibility but 1 sec is a bit too
>>>>> much.
>>>>>
>>>>>  Thanks,
>>>>> +Vinod
>>>>>
>>>>> On Oct 21, 2013, at 7:16 AM, Krishna Kishore Bonagiri wrote:
>>>>>
>>>>> Hi,
>>>>>   I am seeing the following call to start() on AMRMClientAsync taking
>>>>> from 0.9 to 1 second. Why does it take that long? Is there a way to reduce
>>>>> it, I mean does it depend on any of the interval parameters or so in
>>>>> configuration files? I have tried reducing the value of the first argument
>>>>> below from 1000 to 100 seconds also, but that doesn't help.
>>>>>
>>>>>     AMRMClientAsync.CallbackHandler allocListener = new
>>>>> RMCallbackHandler();
>>>>>     amRMClient = AMRMClientAsync.createAMRMClientAsync(1000,
>>>>> allocListener);
>>>>>     amRMClient.init(conf);
>>>>>     amRMClient.start();
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Kishore
>>>>>
>>>>>
>>>>>
>>>>> CONFIDENTIALITY NOTICE
>>>>> NOTICE: This message is intended for the use of the individual or
>>>>> entity to which it is addressed and may contain information that is
>>>>> confidential, privileged and exempt from disclosure under applicable law.
>>>>> If the reader of this message is not the intended recipient, you are hereby
>>>>> notified that any printing, copying, dissemination, distribution,
>>>>> disclosure or forwarding of this communication is strictly prohibited. If
>>>>> you have received this communication in error, please contact the sender
>>>>> immediately and delete it from your system. Thank You.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Alejandro
>>>>
>>>
>>>
>>
>
>
> --
> Alejandro
>