Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> [unit tests] small tests vs. others


+
N Keywal 2011-10-20, 20:46
+
Todd Lipcon 2011-10-20, 20:48
+
Jonathan Gray 2011-10-20, 21:33
Copy link to this message
-
Re: [unit tests] small tests vs. others
+1 on small/medium/large

I have a test mvn repo up on people.apache.org that we can use for a
patched surefire build, if that helps.
On Thu, Oct 20, 2011 at 2:33 PM, Jonathan Gray <[EMAIL PROTECTED]> wrote:
> +1 on small/medium/large.
>
>> -----Original Message-----
>> From: Todd Lipcon [mailto:[EMAIL PROTECTED]]
>> Sent: Thursday, October 20, 2011 1:49 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: [unit tests] small tests vs. others
>>
>> +1 on separating into small/medium/large. One of the requirements of
>> "small" should be that it can run parallel with other tests and in nofork mode.
>>
>> See SUREFIRE-329 for the necessary surefire patch. Until that's integrated
>> upstream we'd have to have our own patched surefire :(
>>
>> -Todd
>>
>> On Thu, Oct 20, 2011 at 1:46 PM, N Keywal <[EMAIL PROTECTED]> wrote:
>> > Hi,
>> >
>> > I've done a stat on the tests we have. Many of them are actually quite
>> > simple: There are 226 test classes in HBase.  100  of them does not
>> > use a cluster, and takes less than 2 seconds to execute. 24 do not use
>> > a cluster but need between 2 and 40 seconds.
>> >
>> > I tried to execute these with different maven settings.
>> > fork: once or always, i.e. all the tests are executed in the same JVM
>> > or there is a new JVM created for every test class parallel;
>> > deactivated or activated, with 3 threads and a per class
>> > parallelisation.
>> >
>> > The 100 tests executes for the whole maven stuff, including the
>> > initial dependency & task analysis fork always: 1m50s fork once: 30
>> > seconds
>> > parallel: 30s (=> no difference).
>> >
>> > It works quite well, except with TestLruBlockCache that I had to remove.
>> >
>> > I tried as well with the 24 'bigger' ones but no cluster, for the
>> > following
>> > results:
>> > fork always: 7m36s
>> > parallel: 2m32s: 3 times faster.
>> >
>> > However, I had to remove many of them, there are some conflicts that I
>> > have not solved (nor tried to).
>> >
>> > The fact that nearly 50% of the tests can be executed in 1 minutes is
>> > interesting imho.
>> >
>> > That makes me wonder if we should try to get a separation of the tests
>> > in 3 parts, small/medium/large as of
>> > http://googletesting.blogspot.com/2009/07/software-testing-categorizat
>> > ion.html
>> >
>> > small would be:
>> > - less than two seconds, can be executed in the same jvm
>> > - all of them: less than 2 minute
>> > - no cluster, nothing that can break the environment
>> > - run by the developers as a part of the build
>> >
>> > medium:
>> > - less than 40 seconds
>> > - all of them: would last less than 30 minutes, ideally 15 minutes
>> > - run in an isolated jvm
>> > - run explicitly by the developers before submitting a fix
>> >
>> > large:
>> > - run in an isolated jvm
>> > - Should last less than 45minutes
>> > - run on the central built or explicitly by the developers
>> > - break the built when it fails.
>> >
>> > Only one issue: it's not directly possible with maven, but there are
>> > known workarounds :-)
>> >
>> >
>> > What do you think?
>> >
>> >
>> > Below the tests set for the 120 tests that seems to work altogether.
>> >
>> >
>> > mvn test
>> > -
>> Dtest=org.apache.hadoop.hbase.avro.TestAvroUtil,org.apache.hadoop.hba
>> > se.client.TestAttributes,org.apache.hadoop.hbase.client.TestGet,org.ap
>> > ache.hadoop.hbase.client.TestOperation,org.apache.hadoop.hbase.client.
>> > TestResult,org.apache.hadoop.hbase.client.TestScan,org.apache.hadoop.h
>> >
>> base.coprocessor.TestRegionObserverStacking,org.apache.hadoop.hbase.fi
>> > lter.TestBitComparator,org.apache.hadoop.hbase.filter.TestColumnPagina
>> > tionFilter,org.apache.hadoop.hbase.filter.TestDependentColumnFilter,or
>> > g.apache.hadoop.hbase.filter.TestFilterList,org.apache.hadoop.hbase.fi
>> > lter.TestInclusiveStopFilter,org.apache.hadoop.hbase.filter.TestPageFi
>> > lter,org.apache.hadoop.hbase.filter.TestParseFilter,org.apache.hadoop.
>> > hbase.filter.TestPrefixFilter,org.apache.hadoop.hbase.filter.TestRando
+
Stack 2011-10-20, 20:56
+
Andrew Purtell 2011-10-20, 22:04
+
Akash Ashok 2011-10-21, 01:17
+
Gary Helmling 2011-11-14, 19:47
+
N Keywal 2011-11-14, 20:25
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB