Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Time taken for starting AMRMClientAsync


+
Krishna Kishore Bonagiri 2013-10-21, 14:16
+
Alejandro Abdelnur 2013-10-21, 15:53
Copy link to this message
-
Re: Time taken for starting AMRMClientAsync
Hi Alejandro,

  I submit all my applications from a single Client, but all of my
application masters are taking almost the same amount of time for finishing
the above calls. Do you reuse ApplicationMaster instances or do some other
thing for saving this time? Otherwise I felt the fresh application
connecting to the resource manager would take the same amount of time
although I don't know why should it take that much?

Thanks,
Kishore
On Mon, Oct 21, 2013 at 9:23 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:

> Hi Krishna,
>
> Those 900ms seems consistent with the numbers we  found while doing some
> benchmarks in the context of Llama:
>
> http://cloudera.github.io/llama/
>
> "We found that the first application master created from a client process
> takes around 900 ms to be ready to submit resource requests. Subsequent
> application masters created from the same client process take a mean of 20
> ms. The application master submission throughput (discarding the first
> submission) tops at approximately 100 application masters per second."
>
> I believe there is room for improvement there.
>
> Cheers
>
>
> On Mon, Oct 21, 2013 at 7:16 AM, Krishna Kishore Bonagiri <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>>   I am seeing the following call to start() on AMRMClientAsync taking
>> from 0.9 to 1 second. Why does it take that long? Is there a way to reduce
>> it, I mean does it depend on any of the interval parameters or so in
>> configuration files? I have tried reducing the value of the first argument
>> below from 1000 to 100 seconds also, but that doesn't help.
>>
>>     AMRMClientAsync.CallbackHandler allocListener = new
>> RMCallbackHandler();
>>     amRMClient = AMRMClientAsync.createAMRMClientAsync(1000,
>> allocListener);
>>     amRMClient.init(conf);
>>     amRMClient.start();
>>
>>
>> Thanks,
>> Kishore
>>
>>
>
>
> --
> Alejandro
>
+
Vinod Kumar Vavilapalli 2013-11-17, 19:16
+
Alejandro Abdelnur 2013-11-18, 18:45
+
Krishna Kishore Bonagiri 2013-11-19, 12:37
+
Krishna Kishore Bonagiri 2013-11-21, 14:11
+
Alejandro Abdelnur 2013-11-26, 04:36