Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - fastest way to run tests


+
Matt Corgan 2012-08-26, 22:57
+
N Keywal 2012-08-27, 06:01
+
Jesse Yates 2012-08-27, 06:41
+
Lars George 2012-08-27, 06:51
+
N Keywal 2012-08-27, 07:29
+
Ted Yu 2012-08-27, 21:16
+
Stack 2012-08-27, 21:17
+
Matt Corgan 2012-08-28, 00:23
+
Jimmy Xiang 2012-08-28, 03:06
+
N Keywal 2012-08-28, 05:35
Copy link to this message
-
Re: fastest way to run tests
Jimmy Xiang 2012-08-30, 18:50
This is very helpful, thanks!

I played with it some time these days. The testing is much faster,
which is great.

However, there is one issue.  With parallel testing, very likely, some
tests will be
killed. As a result, the build could
     1 either fail,
     2 or succeed with a smaller total number of "Tests run"

I thought these Killed tests hanging. But they pass if tested not in parallel.

Is this a test case issue, or an issue with surefire?

Some of those red jenkins builds are because some tests are killed
as findHangingTest.sh shows them hanging.

I was wondering before showing the build red, can we find those tests, then
run them not in parallel one more time?  This could be done in a jenkin
build script, right?

Thanks,
Jimmy
On Mon, Aug 27, 2012 at 10:35 PM, N Keywal <[EMAIL PROTECTED]> wrote:
> Yes, jps will show the processes when you start the medium & large tests
> (see 15.5.2. Unit Tests in hbase ref guide)
>
> Hadoop-qa builds are executed with 4 processes. If locally you have
> "Address already in use" or ""Filesystem closed" it's a bug or a regression.
> We will need to increase the number of process from time to time to keep a
> reasonable test execution time while adding new tests.
>
> Rules are (section 15 again):
>
>    - As much as possible, tests should be written as category small tests.
>    - All tests must be written to support parallel execution on the same
>    machine, hence they should not use shared resources as fixed ports or fixed
>    file names.
>    - Tests should not overlog. More than 100 lines/second makes the logs
>    complex to read and use i/o that are hence not available for the other
>    tests.
>    - Tests can be written with HBaseTestingUtility. This class offers
>    helper functions to create a temp directory and do the cleanup, or to start
>    a cluster. Categories and execution time
>    - All tests must be categorized, if not they could be skipped.
>    - All tests should be written to be as fast as possible.
>    - Small category tests should last less than 15 seconds, and must not
>    have any side effect.
>    - Medium category tests should last less than 50 seconds.
>    - Large category tests should last less than 3 minutes. This should
>    ensure a good parallelization for people using it, and ease the analysis
>    when the test fails.
>
> Cheers,
>
> Nicolas
>
>
>
> On Tue, Aug 28, 2012 at 5:06 AM, Jimmy Xiang <[EMAIL PROTECTED]> wrote:
>
>> How can I verify that the thread count setting is used?
>>
>> Will jps show that many surefire processes?
>>
>> Thanks,
>> Jimmy
>>
>> On Mon, Aug 27, 2012 at 5:23 PM, Matt Corgan <[EMAIL PROTECTED]> wrote:
>> > Thanks  - that helps a good bit.  I do get some failures from things like
>> > "Filesystem closed" and "Address already in use", but sounds like that's
>> > not unexpected right now.
>> >
>> >
>> > On Mon, Aug 27, 2012 at 2:17 PM, Stack <[EMAIL PROTECTED]> wrote:
>> >
>> >> On Mon, Aug 27, 2012 at 2:16 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> >> > Thanks N for sharing the knowledge.
>> >> >
>> >> > Looks like some test needs to use random port(s):
>> >> >
>> >> >
>> >> testStopDuringStart(org.apache.hadoop.hbase.master.TestMasterNoCluster):
>> >> > Problem binding to sea-lab-0/10.249.196.101:60000 : Address already
>> in
>> >> use
>> >> >   testFailover(org.apache.hadoop.hbase.master.TestMasterNoCluster):
>> >> Problem
>> >> > binding to sea-lab-0/10.249.196.101:60000 : Address already in use
>> >> >
>> testCatalogDeploys(org.apache.hadoop.hbase.master.TestMasterNoCluster):
>> >> > Problem binding to sea-lab-0/10.249.196.101:60000 : Address already
>> in
>> >> use
>> >> >
>> >> > Cheers
>> >> >
>> >>
>> >> File an issue Ted?
>> >> St.Ack
>> >>
>>
+
N Keywal 2012-08-30, 21:12