Do we have a list of all MR2 frameworks being worked on currently
beyond MPI and Spark?

JP

On Tue, Aug 30, 2011 at 3:04 AM, Milind Bhandarkar (JIRA)
<[EMAIL PROTECTED]> wrote:
> Hamster: Hadoop And Mpi on the same cluSTER
> -------------------------------------------
>
>                 Key: MAPREDUCE-2911
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2911
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: mrv2
>    Affects Versions: 0.23.0
>         Environment: All Unix-Environments
>            Reporter: Milind Bhandarkar
>            Assignee: Milind Bhandarkar
>             Fix For: 0.23.0
>
>
> MPI is commonly used for many machine-learning applications. OpenMPI (http://www.open-mpi.org/) is a popular BSD-licensed version of MPI. In the past, running MPI application on a Hadoop cluster was achieved using Hadoop Streaming (http://videolectures.net/nipsworkshops2010_ye_gbd/), but it was kludgy. After the resource-manager separation from JobTracker in Hadoop, we have all the tools needed to make MPI a first-class citizen on a Hadoop cluster. I am currently working on the patch to make MPI an application-master. Initial version of this patch will be available soon (hopefully before September 10.) This jira will track the development of Hamster: The application master for MPI.
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>

--
Twitter: @jpatanooga
Solution Architect @ Cloudera
hadoop: http://www.cloudera.com

Those are the two public ones I know of currently...

On Aug 31, 2011, at 7:22 PM, Josh Patterson wrote:

> Do we have a list of all MR2 frameworks being worked on currently
> beyond MPI and Spark?
>
> JP
>
> On Tue, Aug 30, 2011 at 3:04 AM, Milind Bhandarkar (JIRA)
> <[EMAIL PROTECTED]> wrote:
>> Hamster: Hadoop And Mpi on the same cluSTER
>> -------------------------------------------
>>
>>                 Key: MAPREDUCE-2911
>>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2911
>>             Project: Hadoop Map/Reduce
>>          Issue Type: New Feature
>>          Components: mrv2
>>    Affects Versions: 0.23.0
>>         Environment: All Unix-Environments
>>            Reporter: Milind Bhandarkar
>>            Assignee: Milind Bhandarkar
>>             Fix For: 0.23.0
>>
>>
>> MPI is commonly used for many machine-learning applications. OpenMPI (http://www.open-mpi.org/) is a popular BSD-licensed version of MPI. In the past, running MPI application on a Hadoop cluster was achieved using Hadoop Streaming (http://videolectures.net/nipsworkshops2010_ye_gbd/), but it was kludgy. After the resource-manager separation from JobTracker in Hadoop, we have all the tools needed to make MPI a first-class citizen on a Hadoop cluster. I am currently working on the patch to make MPI an application-master. Initial version of this patch will be available soon (hopefully before September 10.) This jira will track the development of Hamster: The application master for MPI.
>>
>> --
>> This message is automatically generated by JIRA.
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
>>
>>
>
>
>
> --
> Twitter: @jpatanooga
> Solution Architect @ Cloudera
> hadoop: http://www.cloudera.com

Is Golden Orb MR2ifying? http://www.goldenorbos.org/

On Wed, Aug 31, 2011 at 9:22 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> Those are the two public ones I know of currently...
>
> On Aug 31, 2011, at 7:22 PM, Josh Patterson wrote:
>
>> Do we have a list of all MR2 frameworks being worked on currently
>> beyond MPI and Spark?
>>
>> JP
>>
>> On Tue, Aug 30, 2011 at 3:04 AM, Milind Bhandarkar (JIRA)
>> <[EMAIL PROTECTED]> wrote:
>>> Hamster: Hadoop And Mpi on the same cluSTER
>>> -------------------------------------------
>>>
>>>                 Key: MAPREDUCE-2911
>>>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2911
>>>             Project: Hadoop Map/Reduce
>>>          Issue Type: New Feature
>>>          Components: mrv2
>>>    Affects Versions: 0.23.0
>>>         Environment: All Unix-Environments
>>>            Reporter: Milind Bhandarkar
>>>            Assignee: Milind Bhandarkar
>>>             Fix For: 0.23.0
>>>
>>>
>>> MPI is commonly used for many machine-learning applications. OpenMPI (http://www.open-mpi.org/) is a popular BSD-licensed version of MPI. In the past, running MPI application on a Hadoop cluster was achieved using Hadoop Streaming (http://videolectures.net/nipsworkshops2010_ye_gbd/), but it was kludgy. After the resource-manager separation from JobTracker in Hadoop, we have all the tools needed to make MPI a first-class citizen on a Hadoop cluster. I am currently working on the patch to make MPI an application-master. Initial version of this patch will be available soon (hopefully before September 10.) This jira will track the development of Hamster: The application master for MPI.
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>>
>>>
>>>
>>
>>
>>
>> --
>> Twitter: @jpatanooga
>> Solution Architect @ Cloudera
>> hadoop: http://www.cloudera.com
>
>

--
Todd Lipcon
Software Engineer, Cloudera

https://issues.apache.org/jira/browse/HAMA-431

On Thu, Sep 1, 2011 at 3:13 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> Is Golden Orb MR2ifying? http://www.goldenorbos.org/
>
> On Wed, Aug 31, 2011 at 9:22 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>> Those are the two public ones I know of currently...
>>
>> On Aug 31, 2011, at 7:22 PM, Josh Patterson wrote:
>>
>>> Do we have a list of all MR2 frameworks being worked on currently
>>> beyond MPI and Spark?
>>>
>>> JP
>>>
>>> On Tue, Aug 30, 2011 at 3:04 AM, Milind Bhandarkar (JIRA)
>>> <[EMAIL PROTECTED]> wrote:
>>>> Hamster: Hadoop And Mpi on the same cluSTER
>>>> -------------------------------------------
>>>>
>>>>                 Key: MAPREDUCE-2911
>>>>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2911
>>>>             Project: Hadoop Map/Reduce
>>>>          Issue Type: New Feature
>>>>          Components: mrv2
>>>>    Affects Versions: 0.23.0
>>>>         Environment: All Unix-Environments
>>>>            Reporter: Milind Bhandarkar
>>>>            Assignee: Milind Bhandarkar
>>>>             Fix For: 0.23.0
>>>>
>>>>
>>>> MPI is commonly used for many machine-learning applications. OpenMPI (http://www.open-mpi.org/) is a popular BSD-licensed version of MPI. In the past, running MPI application on a Hadoop cluster was achieved using Hadoop Streaming (http://videolectures.net/nipsworkshops2010_ye_gbd/), but it was kludgy. After the resource-manager separation from JobTracker in Hadoop, we have all the tools needed to make MPI a first-class citizen on a Hadoop cluster. I am currently working on the patch to make MPI an application-master. Initial version of this patch will be available soon (hopefully before September 10.) This jira will track the development of Hamster: The application master for MPI.
>>>>
>>>> --
>>>> This message is automatically generated by JIRA.
>>>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Twitter: @jpatanooga
>>> Solution Architect @ Cloudera
>>> hadoop: http://www.cloudera.com
>>
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

--
Best Regards, Edward J. Yoon
@eddieyoon

On Wed, Aug 31, 2011 at 7:22 PM, Josh Patterson <[EMAIL PROTECTED]> wrote:

> Do we have a list of all MR2 frameworks being worked on currently
> beyond MPI and Spark?
Giraph is also going to port over:

https://issues.apache.org/jira/browse/GIRAPH-13

-- Owen

I've also heard that Matei Z is working on moving Spark to MRv2, but I
havent confirmed that yet.

JP

On Thu, Sep 1, 2011 at 12:22 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> Those are the two public ones I know of currently...
>
> On Aug 31, 2011, at 7:22 PM, Josh Patterson wrote:
>
>> Do we have a list of all MR2 frameworks being worked on currently
>> beyond MPI and Spark?
>>
>> JP
>>
>> On Tue, Aug 30, 2011 at 3:04 AM, Milind Bhandarkar (JIRA)
>> <[EMAIL PROTECTED]> wrote:
>>> Hamster: Hadoop And Mpi on the same cluSTER
>>> -------------------------------------------
>>>
>>>                 Key: MAPREDUCE-2911
>>>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2911
>>>             Project: Hadoop Map/Reduce
>>>          Issue Type: New Feature
>>>          Components: mrv2
>>>    Affects Versions: 0.23.0
>>>         Environment: All Unix-Environments
>>>            Reporter: Milind Bhandarkar
>>>            Assignee: Milind Bhandarkar
>>>             Fix For: 0.23.0
>>>
>>>
>>> MPI is commonly used for many machine-learning applications. OpenMPI (http://www.open-mpi.org/) is a popular BSD-licensed version of MPI. In the past, running MPI application on a Hadoop cluster was achieved using Hadoop Streaming (http://videolectures.net/nipsworkshops2010_ye_gbd/), but it was kludgy. After the resource-manager separation from JobTracker in Hadoop, we have all the tools needed to make MPI a first-class citizen on a Hadoop cluster. I am currently working on the patch to make MPI an application-master. Initial version of this patch will be available soon (hopefully before September 10.) This jira will track the development of Hamster: The application master for MPI.
>>>
>>> --
>>> This message is automatically generated by JIRA.
>>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>>
>>>
>>>
>>
>>
>>
>> --
>> Twitter: @jpatanooga
>> Solution Architect @ Cloudera
>> hadoop: http://www.cloudera.com
>
>

--
Twitter: @jpatanooga
Solution Architect @ Cloudera
hadoop: http://www.cloudera.com

I've started a wiki to track this, please help keep it up-to-date...

http://wiki.apache.org/hadoop/PoweredByYarn

thanks,
Arun

On Aug 31, 2011, at 7:22 PM, Josh Patterson wrote:

> Do we have a list of all MR2 frameworks being worked on currently
> beyond MPI and Spark?
>
> JP
>
> On Tue, Aug 30, 2011 at 3:04 AM, Milind Bhandarkar (JIRA)
> <[EMAIL PROTECTED]> wrote:
>> Hamster: Hadoop And Mpi on the same cluSTER
>> -------------------------------------------
>>
>>                 Key: MAPREDUCE-2911
>>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2911
>>             Project: Hadoop Map/Reduce
>>          Issue Type: New Feature
>>          Components: mrv2
>>    Affects Versions: 0.23.0
>>         Environment: All Unix-Environments
>>            Reporter: Milind Bhandarkar
>>            Assignee: Milind Bhandarkar
>>             Fix For: 0.23.0
>>
>>
>> MPI is commonly used for many machine-learning applications. OpenMPI (http://www.open-mpi.org/) is a popular BSD-licensed version of MPI. In the past, running MPI application on a Hadoop cluster was achieved using Hadoop Streaming (http://videolectures.net/nipsworkshops2010_ye_gbd/), but it was kludgy. After the resource-manager separation from JobTracker in Hadoop, we have all the tools needed to make MPI a first-class citizen on a Hadoop cluster. I am currently working on the patch to make MPI an application-master. Initial version of this patch will be available soon (hopefully before September 10.) This jira will track the development of Hamster: The application master for MPI.
>>
>> --
>> This message is automatically generated by JIRA.
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
>>
>>
>
>
>
> --
> Twitter: @jpatanooga
> Solution Architect @ Cloudera
> hadoop: http://www.cloudera.com