Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Large feature development


Copy link to this message
-
Re: Large feature development
Todd,

On Sep 2, 2012, at 6:12 PM, Todd Lipcon wrote:

> First, let me apologize if my email came off as a personal "snipe"
> against the project or anyone working on it. I know the team has been
> hard at work for multiple years now on the project, and I certainly
> don't mean to denigrate the work anyone has done.
>
> But, I'll stand by my point that YARN is at this point more "alpha"
> than HDFS2.

I'll unfair to tag-team me while consistently ignoring what I write.
(We are also in danger of hitting the threefold repetition rule: http://en.wikipedia.org/wiki/Threefold_repetition. *smile*)
Anyway, I'l repeat, here are the facts on the ground - the work we've done testing/stabilizing YARN/MRv2, it's stability, user-certification across thousands of unique apps, deployment etc. etc.: http://s.apache.org/QVX

> You brought up two bugs in the HDFS2 code base as examples
> of HDFS 2 not being high quality.

Through a lot of words you just agreed with what I said - if people didn't upgrade to HDFS2 (not just HA) they wouldn't hit any of these: HDFS-3626, HDFS-3731 etc. There are more, for e.g. how do folks work around Secondary NN not starting up on upgrades from hadoop-1 (HDFS-3597)? They just copy multiple PBs over to a new hadoop-2 cluster, or patch SNN themselves post HDFS-1073?

Anyway, I agree, we should talk about this in context of an actual release - hadoop-2.1.0 should mark YARN as *beta* IMO - particularly since it will be deployed at scale.

Arun
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB