Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Disable flaky tests, and let Jenkin stay blue?


Copy link to this message
-
Re: Disable flaky tests, and let Jenkin stay blue?
When I tried to find out a way to ignore a single test case, I came across
this:

"Tests *should* be consistent, they *should* be portable, and they
*should*always pass. If a specific test is problematic I would
consider re-writing
it, removing it entirely, or moving it to a different test suite or
project."

http://stackoverflow.com/questions/835705/is-there-a-way-to-skip-only-a-single-test-in-maven
On Tue, Apr 2, 2013 at 10:28 AM, Stack <[EMAIL PROTECTED]> wrote:

> On Tue, Apr 2, 2013 at 10:09 AM, Jimmy Xiang <[EMAIL PROTECTED]> wrote:
>
> > We have not seen couple blue Jenkin builds for 0.95/trunk for quite some
> > time.  Because of this, sometimes we ignore the precommit build failures,
> > which could let some bugs (code or test) sneaked in.
> >
> > I was wondering if it is time to disable all flaky tests and let Jenkin
> > stay blue.  We can maintain a list of tests disabled, and get them back
> > once they are fixed. For each disabled test, if someone wants to get it
> > back, please file a jira so that we don't duplicate the effort and work
> on
> > the same one.
> >
> > As to how to define a test flaky, to me, if a test fails twice in the
> last
> > 10/20 runs, then it is flaky, if there is no apparent env issue.
> >
> > We have different Jenkins job for hadoop 1 and hadoop 2.  If a test is
> > flaky for either one, it is flaky.
> >
> > What do you think?
> >
>
>
> +1
>
> I'd like to help out.
>
> St.Ack
>