Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> [unit tests] small tests vs. others


Copy link to this message
-
RE: [unit tests] small tests vs. others
+1 on small/medium/large.

> -----Original Message-----
> From: Todd Lipcon [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, October 20, 2011 1:49 PM
> To: [EMAIL PROTECTED]
> Subject: Re: [unit tests] small tests vs. others
>
> +1 on separating into small/medium/large. One of the requirements of
> "small" should be that it can run parallel with other tests and in nofork mode.
>
> See SUREFIRE-329 for the necessary surefire patch. Until that's integrated
> upstream we'd have to have our own patched surefire :(
>
> -Todd
>
> On Thu, Oct 20, 2011 at 1:46 PM, N Keywal <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I've done a stat on the tests we have. Many of them are actually quite
> > simple: There are 226 test classes in HBase.  100  of them does not
> > use a cluster, and takes less than 2 seconds to execute. 24 do not use
> > a cluster but need between 2 and 40 seconds.
> >
> > I tried to execute these with different maven settings.
> > fork: once or always, i.e. all the tests are executed in the same JVM
> > or there is a new JVM created for every test class parallel;
> > deactivated or activated, with 3 threads and a per class
> > parallelisation.
> >
> > The 100 tests executes for the whole maven stuff, including the
> > initial dependency & task analysis fork always: 1m50s fork once: 30
> > seconds
> > parallel: 30s (=> no difference).
> >
> > It works quite well, except with TestLruBlockCache that I had to remove.
> >
> > I tried as well with the 24 'bigger' ones but no cluster, for the
> > following
> > results:
> > fork always: 7m36s
> > parallel: 2m32s: 3 times faster.
> >
> > However, I had to remove many of them, there are some conflicts that I
> > have not solved (nor tried to).
> >
> > The fact that nearly 50% of the tests can be executed in 1 minutes is
> > interesting imho.
> >
> > That makes me wonder if we should try to get a separation of the tests
> > in 3 parts, small/medium/large as of
> > http://googletesting.blogspot.com/2009/07/software-testing-categorizat
> > ion.html
> >
> > small would be:
> > - less than two seconds, can be executed in the same jvm
> > - all of them: less than 2 minute
> > - no cluster, nothing that can break the environment
> > - run by the developers as a part of the build
> >
> > medium:
> > - less than 40 seconds
> > - all of them: would last less than 30 minutes, ideally 15 minutes
> > - run in an isolated jvm
> > - run explicitly by the developers before submitting a fix
> >
> > large:
> > - run in an isolated jvm
> > - Should last less than 45minutes
> > - run on the central built or explicitly by the developers
> > - break the built when it fails.
> >
> > Only one issue: it's not directly possible with maven, but there are
> > known workarounds :-)
> >
> >
> > What do you think?
> >
> >
> > Below the tests set for the 120 tests that seems to work altogether.
> >
> >
> > mvn test
> > -
> Dtest=org.apache.hadoop.hbase.avro.TestAvroUtil,org.apache.hadoop.hba
> > se.client.TestAttributes,org.apache.hadoop.hbase.client.TestGet,org.ap
> > ache.hadoop.hbase.client.TestOperation,org.apache.hadoop.hbase.client.
> > TestResult,org.apache.hadoop.hbase.client.TestScan,org.apache.hadoop.h
> >
> base.coprocessor.TestRegionObserverStacking,org.apache.hadoop.hbase.fi
> > lter.TestBitComparator,org.apache.hadoop.hbase.filter.TestColumnPagina
> > tionFilter,org.apache.hadoop.hbase.filter.TestDependentColumnFilter,or
> > g.apache.hadoop.hbase.filter.TestFilterList,org.apache.hadoop.hbase.fi
> > lter.TestInclusiveStopFilter,org.apache.hadoop.hbase.filter.TestPageFi
> > lter,org.apache.hadoop.hbase.filter.TestParseFilter,org.apache.hadoop.
> > hbase.filter.TestPrefixFilter,org.apache.hadoop.hbase.filter.TestRando
> >
> mRowFilter,org.apache.hadoop.hbase.filter.TestSingleColumnValueExclude
> > Filter,org.apache.hadoop.hbase.filter.TestSingleColumnValueFilter,org.
> > apache.hadoop.hbase.io.hfile.slab.TestSlabCache,org.apache.hadoop.hbas
> > e.io.hfile.slab.TestSlab,org.apache.hadoop.hbase.io.hfile.TestBlockCac