Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - Next releases


Copy link to this message
-
Re: Next releases
Sandy Ryza 2013-11-08, 08:15
Starting afresh with 2.2.1 and keeping it as small as possible sounds
reasonable to me.

Would love to get 2.3 out soon.  To that end, how would people feel about
having code and/or feature freeze and/or ship dates?  We've been way behind
out goals for recent releases. Having actual targets on the calendar would
help us achieve the regular release cadence that can really benefit both
downstream projects and ourselves.  Without this, we get in the habit of
stuffing changes into the closest release, even if minor or patch, for fear
that the next will be many months away.

Vinod, Zhijie, or Mayank, please correct me if I'm wrong, but my intuition
is that end-of-year is a little unrealistic for YARN-321.  Will the ~30
working days before the end of the year be enough to complete the feature,
stabilize it, bring APIs to the point that we won't need to break them in
the future, and merge the branch?  Could we either push the feature out or
aim for the end of January instead?

Assuming the latter, some strawman dates:
Feature freeze - 1/6
Code freeze - 1/16
Release - 1/30

Lastly, I'd like to add finer-grained CPU scheduling a la YARN-1089 to the
target features.

thanks,
Sandy
On Thu, Nov 7, 2013 at 6:42 PM, Arun C Murthy <[EMAIL PROTECTED]> wrote:

> Gang,
>
>  Thinking through the next couple of releases here, appreciate f/b.
>
>  # hadoop-2.2.1
>
>  I was looking through commit logs and there is a *lot* of content here
> (81 commits as on 11/7). Some are features/improvements and some are fixes
> - it's really hard to distinguish what is important and what isn't.
>
>  I propose we start with a blank slate (i.e. blow away branch-2.2 and
> start fresh from a copy of branch-2.2.0)  and then be very careful and
> meticulous about including only *blocker* fixes in branch-2.2. So, most of
> the content here comes via the next minor release (i.e. hadoop-2.3)
>
>  In future, we continue to be *very* parsimonious about what gets into a
> patch release (major.minor.patch) - in general, these should be only
> *blocker* fixes or key operational issues.
>
>  # hadoop-2.3
>
>  I'd like to propose the following features for YARN/MR to make it into
> hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
>  * Application History Server - This is happening in  a branch and is
> close; with it we can provide a reasonable experience for new frameworks
> being built on top of YARN.
>  * Bug-fixes in RM Restart
>  * Minimal support for long-running applications (e.g. security) via
> YARN-896
>  * RM Fail-over via ZKFC
>  * Anything else?
>
>  HDFS???
>
>  Overall, I feel like we have a decent chance of rolling hadoop-2.3 by the
> end of the year.
>
>  Thoughts?
>
> thanks,
> Arun
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>