Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Breaking down big unit tests


Copy link to this message
-
Breaking down big unit tests
Hi all,

Apparently some unit test classes are so fat that retesting them is a pain.
While reducing the full testing time is a long-term goal, shall we just
break down those big units into smaller pieces? Here are the running time
of top 20 big units :

3,432.68 org.apache.pig.test.TestEvalPipeline2
2,944.075 org.apache.pig.test.TestSkewedJoin
1,819.059 org.apache.pig.test.TestMergeJoin
1,797.877 org.apache.pig.test.TestFRJoin
1,476.097 org.apache.pig.test.TestEvalPipeline
1,261.661 org.apache.pig.test.TestFRJoin2
1,164.076 org.apache.pig.test.TestAccumulator
801.747 org.apache.pig.test.TestBZip
799.689 org.apache.pig.test.TestJoin
792.808 org.apache.pig.test.TestPigRunner
750.614 org.apache.pig.test.TestStreaming
743.728 org.apache.pig.test.TestNativeMapReduce
739.31 org.apache.pig.test.TestLimitVariable
674.208 org.apache.pig.test.TestScriptLanguageJavaScript
664.857 org.apache.pig.test.TestJoinSmoke
653.671 org.apache.pig.test.TestCounters
621.06 org.apache.pig.test.TestBestFitCast
541.43 org.apache.pig.test.TestAlgebraicEval
539.939 org.apache.pig.test.TestGrunt

While the full tests take about 10 hours to finish, these top 20 classes
account for almost half the time. The idea is to cut them each to 10-minute
pieces. Any comment?

Jie
+
Thejas Nair 2012-07-20, 00:14
+
Julien Le Dem 2012-07-20, 01:31
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB