-[DISCUSS] longer release candidate time for higher quality releases
Benjamin Reed 2012-02-11, 21:09
at the meetup yesterday we discussed/debated ways to make sure we have
high quality releases.
we wanted to put the discussion on the mailing list to see if we
should make a policy change.
MOTIVATION: it is clear that to have high quality releases, we need
quality testing. zookeeper is such a critical service that we can't
afford to put things out that we don't have high confidence in. to get
quality testing we need good test cases, something we agree we all
need to continue working on. this proposal is not put forth as a
substitute for quality test cases.
another thing that some of us put forward is that we need good
pre-release testing from actual users. these are system tests that
zookeeper users run in their qa environments. the great thing about
these tests is that users have such diverse uses and environments that
sometimes they hit cases that sometimes are hard to come up with. they
also have some rather creative uses of zookeeper that we don't always
the current release voting time is 3-5 days. during this time it is
usually the committers and pmc that run their various checks to make
sure the release looks good. however, 3-5 days isn't really long
enough for customers to try things out. actually in that time period
it's possible for people to miss the voting period all together given
that they have their normal work to do.
also, designating a release as an alpha or beta release is a
confusing. does everyone know that 3.4.x is currently a beta release?
PROPOSAL: for non-bugfix releases we should extend the release
candidate time to 2 weeks (more?). the committers and pmc should have
all their votes in in the normal 3 day voting period, but the rest of
the time should be given to allow our users to validate the release
themselves before releasing.
PRO: major bugs tend to crop up very early on in the release, so we
can catch them before the release is out. if we don't have confidence
in a release, we can just extend the candidacy period.
CON: if everyone just waits until the actual release to try things
out, then all we have done is extended the time it takes to do a
so, the questions are: 1) what do people think in general? and 2) if
we do have a release candidate that is out there for a while before
the real release, would you run it through your test environment?