Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Time taken for starting AMRMClientAsync


Copy link to this message
-
Re: Time taken for starting AMRMClientAsync
Hi Alejandro,

  Can you please see if you can answer my question above? I would like to
reduce the time taken by the above calls made by my Application Master, the
way you do.

Thanks,
Kishore
On Tue, Oct 22, 2013 at 3:09 PM, Krishna Kishore Bonagiri <
[EMAIL PROTECTED]> wrote:

> Hi Alejandro,
>
>   I submit all my applications from a single Client, but all of my
> application masters are taking almost the same amount of time for finishing
> the above calls. Do you reuse ApplicationMaster instances or do some other
> thing for saving this time? Otherwise I felt the fresh application
> connecting to the resource manager would take the same amount of time
> although I don't know why should it take that much?
>
> Thanks,
> Kishore
>
>
> On Mon, Oct 21, 2013 at 9:23 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>
>> Hi Krishna,
>>
>> Those 900ms seems consistent with the numbers we  found while doing some
>> benchmarks in the context of Llama:
>>
>> http://cloudera.github.io/llama/
>>
>> "We found that the first application master created from a client process
>> takes around 900 ms to be ready to submit resource requests. Subsequent
>> application masters created from the same client process take a mean of 20
>> ms. The application master submission throughput (discarding the first
>> submission) tops at approximately 100 application masters per second."
>>
>> I believe there is room for improvement there.
>>
>> Cheers
>>
>>
>> On Mon, Oct 21, 2013 at 7:16 AM, Krishna Kishore Bonagiri <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>   I am seeing the following call to start() on AMRMClientAsync taking
>>> from 0.9 to 1 second. Why does it take that long? Is there a way to reduce
>>> it, I mean does it depend on any of the interval parameters or so in
>>> configuration files? I have tried reducing the value of the first argument
>>> below from 1000 to 100 seconds also, but that doesn't help.
>>>
>>>     AMRMClientAsync.CallbackHandler allocListener = new
>>> RMCallbackHandler();
>>>     amRMClient = AMRMClientAsync.createAMRMClientAsync(1000,
>>> allocListener);
>>>     amRMClient.init(conf);
>>>     amRMClient.start();
>>>
>>>
>>> Thanks,
>>> Kishore
>>>
>>>
>>
>>
>> --
>> Alejandro
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB