Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - Re: Large feature development (YARN vs HDFS)


Copy link to this message
-
Re: Large feature development (YARN vs HDFS)
Eric Baldeschwieler 2012-09-03, 21:30

Referring back to Chris M.s thread, this YARN vs HDFS discussion sounds a lot like an umbrella project issue to me.

On Sep 2, 2012, at 3:11 PM, Arun Murthy wrote:

> Eli,
>
> On Sep 2, 2012, at 1:01 PM, Eli Collins <[EMAIL PROTECTED]> wrote:
>
>> On Sat, Sep 1, 2012 at 12:47 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>>> Todd,
>>>
>>> On Sep 1, 2012, at 1:20 AM, Todd Lipcon wrote:
>>>
>>>> I'd actually contend that YARN was merged too early. I have yet to see
>>>> anyone running YARN in production, and it's holding up the "Stable"
>>>> moniker for Hadoop 2.0 -- HDFS-wise we are already quite stable and
>>>> I'm seeing fewer issues in our customers running Hadoop HDFS 2
>>>> compared to Hadoop 1-derived code.
>>>
>>> You know I respect you a ton, but I'm very saddened to see you perpetuate this FUD on our public lists. I expected better, particularly when everyone is working towards the same goals of advancing Hadoop-2. This sniping on other members doing work is, um, I'll just stop here rather than regret later.
>> 2. HDFS is more mature than YARN. Not a surprise given that we all
>> agree YARN is alpha, and a much newer project than HDFS that hasn't
>> yet been deployed in production environments yet (to my knowledge).
>
> Let's focus on the ground reality here.
>
> Please read my (or Rajiv's) message again about YARN's current
> stability and how much it's baked, it's deployment plans to a very
> large cluster in a few *days*. Or, talk to the people developing,
> testing and supporting these customers and clusters.
>
> I'll repeat - YARN has clearly baked much more than HDFS HA given
> the basic bugs (upgrade, edit logs corruption etc.) we've seen after
> being declared *done*; but then we just disagree since clearly I'm
> more conservative. Also, we need to be more conservative wrt HDFS -
> but then what would I know...
>
> I'll admit it's hard to discuss with someone (or a collective) who
> just repeat themselves. Plus, I broke my own rule about email this
> weekend - so, I'll try harder.
>
> Arun