Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - [unit tests] small tests vs. others
N Keywal
2011-10-20, 20:46
Todd Lipcon
2011-10-20, 20:48
Stack
2011-10-20, 20:56
Jonathan Gray
2011-10-20, 21:33
Gary Helmling
2011-10-20, 21:50
Andrew Purtell
2011-10-20, 22:04
Akash Ashok
2011-10-21, 01:17
N Keywal
2011-11-14, 20:25
Copy link to this message
-
Re: [unit tests] small tests vs. others
Gary Helmling 2011-11-14, 19:47
Hi N,

I can push up the modified surefire and junit build with appropriate versions.

Is there anything special to either build that would complicate
building with the supplied patches?  If not, then I can do the builds
and push up the artifacts.  Then we coordinate the testing.  How does
that sound?

--gh
On Mon, Nov 14, 2011 at 5:51 AM, N Keywal <[EMAIL PROTECTED]> wrote:
> Hi Gary,
>
> The hbase specific versions for surefire & junit are ready (see HBASE-4763
> for all the details).
> Would it be possible to use your repository? What would you need from me?
> I've never deployed anything in a maven repo. Surefire works with maven, so
> it should be easy, JUnit does not, so there could be more work to do.
>
> I copy the dev list for information.
>
> Regards,
>
> N.
>
>
> ---------- Forwarded message ----------
> From: Gary Helmling <[EMAIL PROTECTED]>
> Date: Thu, Oct 20, 2011 at 11:50 PM
> Subject: Re: [unit tests] small tests vs. others
> To: [EMAIL PROTECTED]
>
>
> +1 on small/medium/large
>
> I have a test mvn repo up on people.apache.org that we can use for a
> patched surefire build, if that helps.
>
>
> On Thu, Oct 20, 2011 at 2:33 PM, Jonathan Gray <[EMAIL PROTECTED]> wrote:
>> +1 on small/medium/large.
>>
>>> -----Original Message-----
>>> From: Todd Lipcon [mailto:[EMAIL PROTECTED]]
>>> Sent: Thursday, October 20, 2011 1:49 PM
>>> To: [EMAIL PROTECTED]
>>> Subject: Re: [unit tests] small tests vs. others
>>>
>>> +1 on separating into small/medium/large. One of the requirements of
>>> "small" should be that it can run parallel with other tests and in nofork
>>> mode.
>>>
>>> See SUREFIRE-329 for the necessary surefire patch. Until that's
>>> integrated
>>> upstream we'd have to have our own patched surefire :(
>>>
>>> -Todd
>>>
>>> On Thu, Oct 20, 2011 at 1:46 PM, N Keywal <[EMAIL PROTECTED]> wrote:
>>> > Hi,
>>> >
>>> > I've done a stat on the tests we have. Many of them are actually quite
>>> > simple: There are 226 test classes in HBase.  100  of them does not
>>> > use a cluster, and takes less than 2 seconds to execute. 24 do not use
>>> > a cluster but need between 2 and 40 seconds.
>>> >
>>> > I tried to execute these with different maven settings.
>>> > fork: once or always, i.e. all the tests are executed in the same JVM
>>> > or there is a new JVM created for every test class parallel;
>>> > deactivated or activated, with 3 threads and a per class
>>> > parallelisation.
>>> >
>>> > The 100 tests executes for the whole maven stuff, including the
>>> > initial dependency & task analysis fork always: 1m50s fork once: 30
>>> > seconds
>>> > parallel: 30s (=> no difference).
>>> >
>>> > It works quite well, except with TestLruBlockCache that I had to
>>> > remove.
>>> >
>>> > I tried as well with the 24 'bigger' ones but no cluster, for the
>>> > following
>>> > results:
>>> > fork always: 7m36s
>>> > parallel: 2m32s: 3 times faster.
>>> >
>>> > However, I had to remove many of them, there are some conflicts that I
>>> > have not solved (nor tried to).
>>> >
>>> > The fact that nearly 50% of the tests can be executed in 1 minutes is
>>> > interesting imho.
>>> >
>>> > That makes me wonder if we should try to get a separation of the tests
>>> > in 3 parts, small/medium/large as of
>>> > http://googletesting.blogspot.com/2009/07/software-testing-categorizat
>>> > ion.html
>>> >
>>> > small would be:
>>> > - less than two seconds, can be executed in the same jvm
>>> > - all of them: less than 2 minute
>>> > - no cluster, nothing that can break the environment
>>> > - run by the developers as a part of the build
>>> >
>>> > medium:
>>> > - less than 40 seconds
>>> > - all of them: would last less than 30 minutes, ideally 15 minutes
>>> > - run in an isolated jvm
>>> > - run explicitly by the developers before submitting a fix
>>> >
>>> > large:
>>> > - run in an isolated jvm
>>> > - Should last less than 45minutes
>>> > - run on the central built or explicitly by the developers
>>> > - break the built when it fails.