Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - scoping integration tests


Copy link to this message
-
Re: scoping integration tests
Jesse Yates 2011-12-02, 06:49
Sorry, about the message with title: "dev Digest of: thread.25579", my
email has been acting up and I forgot to change the subject line when
pulling from the archives.

I'm reposting this to the right thread for the archives and in case anyone
missed it.

-Jesse

On Thu, Dec 1, 2011 at 6:34 PM, Jesse Yates <[EMAIL PROTECTED]> wrote:

> I'm generally on the same page here, but this is what I'm (currently) thinking about how it could work:
>
>
>
> We need to characterize the unit tests because some are longer than others, and we want to split them up in order of complexity.
>
> Keywal, this next bit is divergent from what I was saying the other day...
>
> The next level up should be testing between integration of components. Again this should be run by the CI; really this is more of a finer grain separation of what was previously just unit tests.
> I've revised my thinking such that this could a different category of unit tests (maybe @Integration that run in their own jvm, in a different step).
> This helps separate concerns between the individual class testing and the testing of multiple pieces working together.
>
>
> Onto using the failsafe plugin.
>
> I like the idea of really leveraging the failsafe plugin to spin up the tests on some of 'real' instance. The (canonical) simple example of using failsafe is to spin up a Jetty server and test whatever stuff you are doing with a 'real'
>
>
> Also, integrating with BigTop to do some of that stuff would also be pretty sweet (and good for both communities).
>
> A good place for these tests would be the hbase-it module discussed in HBASE-4336, since that module can just depend on the rest of the modules (giving full access to the components). Within that package, we could conceivably have the multi-item integration tests (@Integration form above) and then the 'real cluster' tests as well.
>
>
> +1 Having tests that just use the top level API (e.g. what the FB guys are using on their dev cluster after running the full test suite) is really important to make sure we 'real' test cases to ensure functionality.
>
>
> Also, agreeing with Roman, making the plugin to enable that is going to rough, but we should be able to at least find a way to (at least manually at first) have those api level tests run on a real cluster, even if it is a company X and they just post the results. Even then a weekly build/run of that suite could be sufficient at first (though having that for every commit would be must better - anyone know someone at Amazon we can scrounge credits from?).
>
>
> However, I do think that if tests start failing in _any_ of the above phases then the patch that caused it needs to not be committed OR the tests need to be revised (which would also be part of the patch, so my original point stands). That is indicative of broken functionality and trunk should be kept as stable as possible.
>
>
> Yeah, devs shouldn't have to run the full set of tests before submitting a patch, but they should easily be able to run everything up to (but not including) the api-level testing.
> What would be cool though would be that they could even run the api-level testing, but that is just runs on a local MiniCluster, but that's a lot of work out :).
> A lot of that stability comes form having a staged testing cycle as well as helping devs cut down their testing cycles, so anything we do to help cut that time down is good stuff in my book..
>
> -- Jesse
> -------------------
>
>
> ps. I hate to be 'that guy' but we need a consistent way of describing the tests we talking about, but lets push that off until figure out what the heck we actually want to do.
>
>
> Jesse Yates
> 240-888-2200
> @jesse_yates
>
>
>
>> ---------- Forwarded message ----------
>> From: Ted Yu <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Date: Mon, 28 Nov 2011 12:23:43 -0800
>> Subject: scoping integration tests
>> The following discussion is closely related to HBASE-4712.
>>
>> We should reach general consensus so that the execution of future test