-Re: scoping integration tests
Jesse Yates 2011-11-30, 06:28
+1 Overall idea.
I don't know if we should consider the integration tests as the primary
thing that bigtop runs. Since they are deploying to a small cluster
(right?), the tests that FB is supposed to be open sourcing soon should
probably be what they run (and lets call those acceptance tests).
While integration tests could be run by bigtop, they should definitely be
part of our testing of patches as well - no patches should go into trunk
that don't pass both the unit tests and integration tests. Then we push in
bigger patches or cut releases we make sure that the acceptance tests pass.
At least, that's what I was thinking about how testing works.
On Tue, Nov 29, 2011 at 10:08 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> From Stack:
> I think we can have it so everyone gets what they need;
> small/medium/large are done by surefire and then we have a new
> intergration test green field that is run by failsafe that can help us
> get tests for bigtop et al. to run.
> I think the above proposal makes sense.
> Please share your comments.
> On Mon, Nov 28, 2011 at 12:23 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> The following discussion is closely related to HBASE-4712.
>> We should reach general consensus so that the execution of future test
>> strategy is smooth.
>> On Mon, Nov 28, 2011 at 11:50 AM, Jesse Yates <[EMAIL PROTECTED]>wrote:
>>> I was considering 'integration tests' as a separate concern from the
>>> large/medium/small _unit_ tests.
>>> That is, in fact, why the failsafe plugin was added (and is designed
>>> Currently, we have a lot of tests that fall in the realm of integration
>>> tests (testing integration between various pieces, rather than single
>>> functionality alone). The most obvious indicator that something is an
>>> integration test is when it is using the MiniHBaseCluster since that is
>>> really testing the integration between the new feature and the rest of the
>>> The surefire plugin should really be testing unit tests, and then those
>>> can be classified as small/medium/large. However, the large would be
>>> _extremely_ rare cases.
>>> If the community is completely moving to just using the annotations for
>>> denoting size of tests, rather than using the integrationTest/unit test
>>> stuff, thats fine. However, I still think that if we do that, we should
>>> have a separate designation for @IntegrationTest (but that is really
>>> already covered in the fact that they would be named differently and
>>> therefore run by the different plugin). The reason we need to split them
>>> out is to have that separate tier of testing - once they pass unit level,
>>> they can be tested at the integration level and then on the acceptance test
>>> level (though we don't have any of those yet).
>>> This probably means more reclassification of tests.
>>> Making this explicit is important so people know at what point in the
>>> process their patch breaks (as well as staging the testing being good
>>> practice in general). Therefore, the small/medium/large classifications are
>>> not complete. Further, we should should still be able to use the general
>>> maven testing (mvn test, mvn verify) to run all the tests as expected,
>>> rather than having to know which things extra commands to run. To that end,
>>> we need to integrate HBASE-4712<https://issues.apache.org/jira/browse/HBASE-4712>ASAP so its in the book and people know how to work with the current
>>> testing layout.
>>> I've been a bit behind on the testing discussions on dev@, so didn't
>>> get to put these thoughts in, but I think they can still be added on. This
>>> was also part of what Doug, Stack and I were kind of envisioning in the
>>> original email to the list.
>>> On Mon, Nov 28, 2011 at 11:29 AM, N Keywal <[EMAIL PROTECTED]> wrote:
>>>> Today's tests are:
>>>> - 416 small tests, executed in ~3 minutes