Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Breaking down big unit tests


+
Jie Li 2012-07-19, 21:11
Copy link to this message
-
Re: Breaking down big unit tests
We certainly need to look at ways to reduce the runtime of the 'unit'
tests. Some of them should be migrated to the e2e tests.

But what you want for being able to re-test easily seems to be a way to
specify specific test case within a Test*.java file . I wonder if junit
lets you do that.

-Thejas
On 7/19/12 2:11 PM, Jie Li wrote:
> Hi all,
>
> Apparently some unit test classes are so fat that retesting them is a pain.
> While reducing the full testing time is a long-term goal, shall we just
> break down those big units into smaller pieces? Here are the running time
> of top 20 big units :
>
> 3,432.68 org.apache.pig.test.TestEvalPipeline2
> 2,944.075 org.apache.pig.test.TestSkewedJoin
> 1,819.059 org.apache.pig.test.TestMergeJoin
> 1,797.877 org.apache.pig.test.TestFRJoin
> 1,476.097 org.apache.pig.test.TestEvalPipeline
> 1,261.661 org.apache.pig.test.TestFRJoin2
> 1,164.076 org.apache.pig.test.TestAccumulator
> 801.747 org.apache.pig.test.TestBZip
> 799.689 org.apache.pig.test.TestJoin
> 792.808 org.apache.pig.test.TestPigRunner
> 750.614 org.apache.pig.test.TestStreaming
> 743.728 org.apache.pig.test.TestNativeMapReduce
> 739.31 org.apache.pig.test.TestLimitVariable
> 674.208 org.apache.pig.test.TestScriptLanguageJavaScript
> 664.857 org.apache.pig.test.TestJoinSmoke
> 653.671 org.apache.pig.test.TestCounters
> 621.06 org.apache.pig.test.TestBestFitCast
> 541.43 org.apache.pig.test.TestAlgebraicEval
> 539.939 org.apache.pig.test.TestGrunt
>
> While the full tests take about 10 hours to finish, these top 20 classes
> account for almost half the time. The idea is to cut them each to 10-minute
> pieces. Any comment?
>
> Jie
>
+
Julien Le Dem 2012-07-20, 01:31