-Re: Project announcement: Sailfish (also, looking for colloborators)
This would seem like a perfect use case for YARN. Is that what you are
thinking? You could implement this as a new framework rather then
trying to incrementally change map-reduce.
This would let you move faster and demonstrate side by side
E14 - typing on glass
On May 8, 2012, at 10:32 AM, Sriram Rao <[EMAIL PROTECTED]> wrote:
> I'd like to announce the release of a new open source project, Sailfish.
> Sailfish tries to improve Hadoop-performance, particularly for large-jobs
> which process TB's of data and run for hours. In building Sailfish, we
> modify how map-output is handled and transported from map->reduce.
> The project pages provide more information about the project.
> We are looking for colloborators who can help get some of the ideas into
> Apache Hadoop. A possible step forward could be to make "shuffle" phase of
> Hadoop pluggable.
> If you are interested in working with us, please get in touch with me.