Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> unit tests


Copy link to this message
-
Re: unit tests
Alternatively, a good place for this would also be in the book.xml, under
(13) Building and Developing HBase

-Jesse

On Mon, Nov 28, 2011 at 3:10 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> Haven't had a chance to review the full note below, but one request:
> can we please put this in the code tree itself in a file like
> TESTING.txt or README.testing.txt or something?
>
> It'd be nice to be able to refer back to it from within the code tree.
>
> -Todd
>
> On Mon, Nov 28, 2011 at 12:29 PM, N Keywal <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > The surefire modification seems to work well, so we can move to the final
> > step.
> > Please find below the last proposition for the test. The behavior
> described
> > can be activated with:
> >  - a commit of hbase-4847
> >  - a modification of the jenkins config to run mvn verify vs. mvn tests.
> >
> > The main points are:
> >  - should we run small & medium tests only by default on a developer
> > machine => here answer is yes
> >  - should we run all tests on the central build => here answer is yes
> >  - should we stop when a small test fails => here answer is yes
> >  - should we stop when a medium test fails => here answer is yes
> >
> >
> > Also, I just had a discussion with Jesse, who think we should keep/have
> the
> > notion of integration test. If retained, the points above must be
> impacted
> > to take this into account.
> >
> >
> > fyi, there are today:
> >  - 416 small tests, executed in ~3 minutes
> >  - 489 medium tests, executed in ~35 minutes (without parallelization)
> >  - 280 large tests, executed in ~90 minutes (without parallelization)
> >
> > --
> > 1) Running tests
> > HBase tests are divided in three categories: small, medium and large,
> with
> > corresponding JUnit categories: SmallTests, MediumTests, LargeTests.
> >
> > - Small tests are executed in a shared JVM. We put in this category all
> the
> > tests that can be executed quickly (the maximum execution time for a test
> > is 15 seconds, and they do not use a cluster) in a shared jvm.
> > - Medium tests represents tests that must be executed before proposing a
> > patch. They are designed to run in less than 30 minutes altogether, and
> are
> > quite stable in their results. They're designed to last less than 50
> > seconds individually. They can use a cluster, and each of them is
> executed
> > in a separate JVM.
> > - Large tests are everything else. They are typically integration tests,
> > regression tests for specific bugs, timeout tests, performance tests.
> Some
> > of them can be flaky. They are executed before a commit on the
> > pre-integration machines. They can be run on the developer machine as
> well.
> >
> > Commands are:
> > 1) mvn test
> >  - execute small tests in a single JVM and medium tests in a separate JVM
> > for each test
> >  - medium tests are NOT executed if there is an error in a small test
> >  - large tests are NOT executed
> >  - there is one report for small tests, and one report for medium tests
> -if
> > they are executed-
> >
> > 2) mvn verify
> >  - execute small tests in a single JVM then medium tests in a separate
> JVM
> > for each test, then large tests in a separate JVM as well.
> >  - medium tests are NOT executed if there is an error in a small test
> >  - large tests are NOT executed if there is an error in a small or medium
> > test
> >  - there is one report by test category, small, medium and large
> >
> > 3) mvn test -P localTests -Dtest=myTests
> >  - remove any category effect (without this specific profile, the
> profiles
> > are taken into account)
> >  - use actually the official release of surefire & the old connector to
> > junit
> >  - tests are executed in separated JVM
> > - You will see a new message at the end of the report: "[INFO] Tests are
> > skipped". It's harmless.
> >
> > 4) mvn test -P runAllTests
> >  - execute small tests in a single JVM then medium & large tests in a
> > separate JVM for each test
> >  - medium and large tests are NOT executed if there is an error in a

Jesse Yates
240-888-2200
@jesse_yates