Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Can container requests be made paralelly from multiple threads


Copy link to this message
-
Re: Can container requests be made paralelly from multiple threads
Jian He 2013-09-29, 22:21
Hi,

If you adopt hadoop-2.1-beta,  StartContainerRequest is changed to
StartContainer(s)Request, meaning it can accept a list of container start
requests and start those containers in one RPC call.

Jian
On Fri, Sep 27, 2013 at 1:06 PM, Omkar Joshi <[EMAIL PROTECTED]> wrote:

> My point is why you want multiple threads as a part of single AM talking
> to RM simultaneously? I think AMRMProtocol only AM is suppose to use and if
> the requirement is to have multiple requestor requesting resources then it
> should be clubbed into one single request and sent to RM. One more thing
> may be related to this. When AM makes a request to RM; today it gets
> resources only if earlier NM heartbeat resulted into RM scheduling one for
> it. So multiplexing AMRM requests wont help anyway but it will only
> complicate things on RM side. Scheduler is not kicked in (synchronously)
> when AM makes a request.
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Fri, Sep 27, 2013 at 11:14 AM, Krishna Kishore Bonagiri <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Omkar,
>>
>>   Thanks for the quick reply. I have a requirement for sets of containers
>> depending on some of my business logic. I found that each of the request
>> allocations is taking around 2 seconds, so I am thinking of doing the
>> requests at the same from multiple threads.
>>
>> Kishore
>>
>>
>> On Fri, Sep 27, 2013 at 11:27 PM, Omkar Joshi <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>
>>> I suggest you should not do that. After YARN-744 goes in this will be
>>> prevented on RM side. May I know why you want to do this? any advantage/
>>> use case?
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Fri, Sep 27, 2013 at 8:31 AM, Krishna Kishore Bonagiri <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Hi,
>>>>
>>>>   Can we submit container requests from multiple threads in parallel to
>>>> the Resource Manager?
>>>>
>>>> Thanks,
>>>> Kishore
>>>>
>>>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>
>>
>>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.