Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - How to write/run MPI program on Yarn?


Copy link to this message
-
Re: How to write/run MPI program on Yarn?
sam liu 2013-07-01, 02:48
Thanks Pramod and Clark!

1. What's the relationship of Hadoop 2.x branch and mpich2-yarn project?
2. Does Hadoop 2.x branch plan to include MPI implementation? I mentioned
there is already a JIRA:
https://issues.apache.org/jira/browse/MAPREDUCE-2911
2013/7/1 Clark Yang (杨卓荦) <[EMAIL PROTECTED]>

> Hi, sam
>
>    Please try it following the README.md on
> https://github.com/clarkyzl/mpich2-yarn/.
>   The design is simple ,make each nodemanger runs a "smpd" daemon and
> make application master run "mpiexec" to submit mpi jobs. The biggest
> problem here, I think, is the yarn scheduler do not support gang
> secheduling algorithm, and the scheduling is very very slow.
>
>   Any ideas please feel free to share.
>
> Cheers,
> Zhuoluo (Clark) Yang
>
>
> 2013/6/30 Pramod N <[EMAIL PROTECTED]>
>
>> Hi Sam,
>> This might interest you.
>> https://github.com/clarkyzl/mpich2-yarn
>>
>>
>>
>> Pramod N <http://atmachinelearner.blogspot.in>
>> Bruce Wayne of web
>> @machinelearner <https://twitter.com/machinelearner>
>>
>> --
>>
>>
>> On Sun, Jun 30, 2013 at 6:02 PM, sam liu <[EMAIL PROTECTED]> wrote:
>>
>>> Hi Experts,
>>>
>>> Does Hadoop 2.0.5-alpha supports MPI programs?
>>>
>>> - If yes, is there any example of writing a MPI program on Yarn? How to
>>> write the client side code to configure and submit the MPI job?
>>> - If no, which Hadoop version will support MPI programming?
>>>
>>> Thanks!
>>>
>>
>>
>