This is from someone from Hadoop and who's been on and off in Hive.
Dedicated test resources is good, but there are other (simpler?) things worth pursuing to begin with - suggestions from the peanut gallery:
- Split the project into modules. Without thinking much, a simple split could be client, execution engine, metastore. We did the module split in Hadoop, it is initially a bit of pain but pays back a lot in future. And whenever there are isolated module changes, only those modules needs to be tested. Also has the added benefit of clear modularity.
- A separate candidate suite of pre-commit tests. It can be a subset of all the tests, may be even hand-picked. Sure they won't catch some bugs, but it is a reasonable compromise that worked in Hadoop.
- And wire the pre-commit tests with JIRA/Jenkins.
On Jun 16, 2013, at 11:02 AM, Edward Capriolo wrote:
> Hive's unit test suite has gotten larger as we have added more features and
> thus it takes longer to run. For a single machine duel core with solid
> state disks I have to start a test run at night, and then check the next
> morning to see if the run has finished. (I have been running tests for
> maybe 2 hours and am up to escape.q)
> Also for a long time the distribution of which features get reviewed,
> tested, and committed has been unfair. With more people involved in the
> project this situation has gotten better however it is still not fair. What
> sometimes ends up happening is that a good feature, which is reviewed, and
> +1ed sits uncommitted for months or years.
> Some committers or groups of commiters have an agenda and dedicated testing
> resources, and others do not. This unbalances the project. It means that
> small incremental improvements and new features not important to 'large
> company with testing resources x' sit ready to be committed while other
> people working in pairs further the project to their agenda. (This last
> statement is not a condemnation of anyone, just possibly a fact of life)
> 1) The project should sponsor an open and independent build/test farm
> 2) Once a ticket is marked 'patch available' this build farm should
> automatically notice this and begin testing the patch
> 3) patches/issues which pass tests first should be considered 1st for
> We can use a hosted testing service such as:
> Q. Do any committers/interested parties like the idea?
> Q. Would anyone be interested in dedicating financial resources to getting
> this off the ground (I am)
> Q. Does anyone have ideas for a better platform or a better system