Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Re: How to write/run MPI program on Yarn?


Copy link to this message
-
Re: How to write/run MPI program on Yarn?
Any one could help answer above questions? Thanks a lot!
2013/7/1 sam liu <[EMAIL PROTECTED]>

> Thanks Pramod and Clark!
>
> 1. What's the relationship of Hadoop 2.x branch and mpich2-yarn project?
> 2. Does Hadoop 2.x branch plan to include MPI implementation? I mentioned
> there is already a JIRA:
> https://issues.apache.org/jira/browse/MAPREDUCE-2911
>
>
> 2013/7/1 Clark Yang (杨卓荦) <[EMAIL PROTECTED]>
>
> Hi, sam
>>
>>    Please try it following the README.md on
>> https://github.com/clarkyzl/mpich2-yarn/.
>>   The design is simple ,make each nodemanger runs a "smpd" daemon and
>> make application master run "mpiexec" to submit mpi jobs. The biggest
>> problem here, I think, is the yarn scheduler do not support gang
>> secheduling algorithm, and the scheduling is very very slow.
>>
>>   Any ideas please feel free to share.
>>
>> Cheers,
>> Zhuoluo (Clark) Yang
>>
>>
>> 2013/6/30 Pramod N <[EMAIL PROTECTED]>
>>
>>> Hi Sam,
>>> This might interest you.
>>> https://github.com/clarkyzl/mpich2-yarn
>>>
>>>
>>>
>>> Pramod N <http://atmachinelearner.blogspot.in>
>>> Bruce Wayne of web
>>> @machinelearner <https://twitter.com/machinelearner>
>>>
>>> --
>>>
>>>
>>> On Sun, Jun 30, 2013 at 6:02 PM, sam liu <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi Experts,
>>>>
>>>> Does Hadoop 2.0.5-alpha supports MPI programs?
>>>>
>>>> - If yes, is there any example of writing a MPI program on Yarn? How to
>>>> write the client side code to configure and submit the MPI job?
>>>> - If no, which Hadoop version will support MPI programming?
>>>>
>>>> Thanks!
>>>>
>>>
>>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB