Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Project announcement: Sailfish (also, looking for colloborators)


Copy link to this message
-
Re: Project announcement: Sailfish (also, looking for colloborators)
Eric Baldeschwieler 2012-05-08, 21:47
This would seem like a perfect use case for YARN. Is that what you are
thinking?  You could implement this as a new framework rather then
trying to incrementally change map-reduce.

This would let you move faster and demonstrate side by side
performance improvements.

---
E14 - typing on glass

On May 8, 2012, at 10:32 AM, Sriram Rao <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I'd like to announce the release of a new open source project, Sailfish.
>
> http://code.google.com/p/sailfish/
>
> Sailfish tries to improve Hadoop-performance, particularly for large-jobs
> which process TB's of data and run for hours.  In building Sailfish, we
> modify how map-output is handled and transported from map->reduce.
>
> The project pages provide more information about the project.
>
> We are looking for colloborators who can help get some of the ideas into
> Apache Hadoop. A possible step forward could be to make "shuffle" phase of
> Hadoop pluggable.
>
> If you are interested in working with us, please get in touch with me.
>
> Sriram