Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - linux containers with Hadoop


Copy link to this message
-
Re: linux containers with Hadoop
Edward Capriolo 2011-09-30, 17:39
On Fri, Sep 30, 2011 at 9:03 AM, bikash sharma <[EMAIL PROTECTED]>wrote:

> Hi,
> Does anyone knows if Linux containers (which are like kernel supported
> virtualization technique for providing resource isolation across
> process/appication) have ever been used with Hadoop to provide resource
> isolation for map/reduce tasks?
> If yes, what could be the up/down sides of such approach and how feasible
> it
> is in the context of Hadoop?
> Any pointers if any in terms of papers, etc would be useful.
>
> Thanks,
> Bikash
>

Previously hadoop launched map reduce tasks as a single user, now with
security tasks can launch as different users in the same OS/VM. I would say
the closest you can to that isolation is the work done with mesos .
http://www.mesosproject.org/