Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: How to write/run MPI program on Yarn?


Copy link to this message
-
Re: How to write/run MPI program on Yarn?
sam liu 2013-07-02, 09:21
Any one could help answer above questions? Thanks a lot!
2013/7/1 sam liu <[EMAIL PROTECTED]>

> Thanks Pramod and Clark!
>
> 1. What's the relationship of Hadoop 2.x branch and mpich2-yarn project?
> 2. Does Hadoop 2.x branch plan to include MPI implementation? I mentioned
> there is already a JIRA:
> https://issues.apache.org/jira/browse/MAPREDUCE-2911
>
>
> 2013/7/1 Clark Yang (杨卓荦) <[EMAIL PROTECTED]>
>
> Hi, sam
>>
>>    Please try it following the README.md on
>> https://github.com/clarkyzl/mpich2-yarn/.
>>   The design is simple ,make each nodemanger runs a "smpd" daemon and
>> make application master run "mpiexec" to submit mpi jobs. The biggest
>> problem here, I think, is the yarn scheduler do not support gang
>> secheduling algorithm, and the scheduling is very very slow.
>>
>>   Any ideas please feel free to share.
>>
>> Cheers,
>> Zhuoluo (Clark) Yang
>>
>>
>> 2013/6/30 Pramod N <[EMAIL PROTECTED]>
>>
>>> Hi Sam,
>>> This might interest you.
>>> https://github.com/clarkyzl/mpich2-yarn
>>>
>>>
>>>
>>> Pramod N <http://atmachinelearner.blogspot.in>
>>> Bruce Wayne of web
>>> @machinelearner <https://twitter.com/machinelearner>
>>>
>>> --
>>>
>>>
>>> On Sun, Jun 30, 2013 at 6:02 PM, sam liu <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi Experts,
>>>>
>>>> Does Hadoop 2.0.5-alpha supports MPI programs?
>>>>
>>>> - If yes, is there any example of writing a MPI program on Yarn? How to
>>>> write the client side code to configure and submit the MPI job?
>>>> - If no, which Hadoop version will support MPI programming?
>>>>
>>>> Thanks!
>>>>
>>>
>>>
>>
>