Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Large feature development

Copy link to this message
Re: Large feature development

On Sep 2, 2012, at 6:12 PM, Todd Lipcon wrote:

> First, let me apologize if my email came off as a personal "snipe"
> against the project or anyone working on it. I know the team has been
> hard at work for multiple years now on the project, and I certainly
> don't mean to denigrate the work anyone has done.
> But, I'll stand by my point that YARN is at this point more "alpha"
> than HDFS2.

I'll unfair to tag-team me while consistently ignoring what I write.
(We are also in danger of hitting the threefold repetition rule: http://en.wikipedia.org/wiki/Threefold_repetition. *smile*)
Anyway, I'l repeat, here are the facts on the ground - the work we've done testing/stabilizing YARN/MRv2, it's stability, user-certification across thousands of unique apps, deployment etc. etc.: http://s.apache.org/QVX

> You brought up two bugs in the HDFS2 code base as examples
> of HDFS 2 not being high quality.

Through a lot of words you just agreed with what I said - if people didn't upgrade to HDFS2 (not just HA) they wouldn't hit any of these: HDFS-3626, HDFS-3731 etc. There are more, for e.g. how do folks work around Secondary NN not starting up on upgrades from hadoop-1 (HDFS-3597)? They just copy multiple PBs over to a new hadoop-2 cluster, or patch SNN themselves post HDFS-1073?

Anyway, I agree, we should talk about this in context of an actual release - hadoop-2.1.0 should mark YARN as *beta* IMO - particularly since it will be deployed at scale.