Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Disable flaky tests, and let Jenkin stay blue?


Copy link to this message
-
Re: Disable flaky tests, and let Jenkin stay blue?
When I tried to find out a way to ignore a single test case, I came across
this:

"Tests *should* be consistent, they *should* be portable, and they
*should*always pass. If a specific test is problematic I would
consider re-writing
it, removing it entirely, or moving it to a different test suite or
project."

http://stackoverflow.com/questions/835705/is-there-a-way-to-skip-only-a-single-test-in-maven
On Tue, Apr 2, 2013 at 10:28 AM, Stack <[EMAIL PROTECTED]> wrote:

> On Tue, Apr 2, 2013 at 10:09 AM, Jimmy Xiang <[EMAIL PROTECTED]> wrote:
>
> > We have not seen couple blue Jenkin builds for 0.95/trunk for quite some
> > time.  Because of this, sometimes we ignore the precommit build failures,
> > which could let some bugs (code or test) sneaked in.
> >
> > I was wondering if it is time to disable all flaky tests and let Jenkin
> > stay blue.  We can maintain a list of tests disabled, and get them back
> > once they are fixed. For each disabled test, if someone wants to get it
> > back, please file a jira so that we don't duplicate the effort and work
> on
> > the same one.
> >
> > As to how to define a test flaky, to me, if a test fails twice in the
> last
> > 10/20 runs, then it is flaky, if there is no apparent env issue.
> >
> > We have different Jenkins job for hadoop 1 and hadoop 2.  If a test is
> > flaky for either one, it is flaky.
> >
> > What do you think?
> >
>
>
> +1
>
> I'd like to help out.
>
> St.Ack
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB