Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # dev - Abstraction layer to support both YARN and Mesos


+
Tsuyoshi OZAWA 2013-07-26, 06:13
+
Harsh J 2013-07-26, 10:14
+
Tsuyoshi OZAWA 2013-07-29, 14:58
+
Steve Loughran 2013-07-31, 15:30
+
Tom White 2013-07-31, 15:40
+
Bikas Saha 2013-07-31, 16:34
+
Vinod Kumar Vavilapalli 2013-07-31, 16:45
Copy link to this message
-
Re: Abstraction layer to support both YARN and Mesos
Tsuyoshi OZAWA 2013-07-29, 15:42
Harsh, yes, I know what you mean :-) Never mind. We should discuss
this topic with MR users.

On Tue, Jul 30, 2013 at 12:08 AM, Michael Segel
<[EMAIL PROTECTED]> wrote:
> Actually,
> I am interested.
>
> Lots of different Apache top level projects seem to overlap and it can be confusing.
> Its very easy for a good technology to get starved because no one asks how to combine these features in to the framework.
>
> On Jul 29, 2013, at 10:06 AM, Michael Segel <[EMAIL PROTECTED]> wrote:
>
>> Actually,
>> I am interested.
>>
>> Lots of different Apache top level projects seem to overlap and it can be confusing.
>> Its very easy for a good technology to get starved because no one asks how to combine these features in to the framework.
>>
>>
>> On Jul 29, 2013, at 9:58 AM, Tsuyoshi OZAWA <[EMAIL PROTECTED]> wrote:
>>
>>> I thought some high availability and resource isolation features in
>>> Mesos are more matured. If no one is interested in this topic, MR
>>> should go with YARN.
>>>
>>> On Fri, Jul 26, 2013 at 7:14 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>>> Do we have a good reason to prefer Mesos over YARN for scheduling MR
>>>> specifically? At what times would one prefer the other?
>>>>
>>>> On Fri, Jul 26, 2013 at 11:43 AM, Tsuyoshi OZAWA
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> Hi,
>>>>>
>>>>> Now, Apache Mesos, an distributed resource manager, is top-level
>>>>> apache project. Meanwhile, As you know, Hadoop has own resource
>>>>> manager - YARN. IMHO, we should make resource manager pluggable in
>>>>> MRv2, because there are their own field users of MapReduce would like
>>>>> to use. I think this work is useful for MapReduce users. On the other
>>>>> hand, this work can also be large, because MRv2's code base is tightly
>>>>> coupled with YARN currently. Thoughts?
>>>>>
>>>>> - Tsuyoshi
>>>>
>>>>
>>>>
>>>> --
>>>> Harsh J
>>>
>>>
>>>
>>> --
>>> - Tsuyoshi
>>>
>>
>

--
- Tsuyoshi