Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Flaky Testcase Detector


Copy link to this message
-
Re: Flaky Testcase Detector
Stack 2013-03-06, 06:43
Nice.  Should we check it into ./dev-tools?
Thanks,
St.Ack
On Tue, Mar 5, 2013 at 7:12 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> This is handy. Thanks so much for sharing it.
>
> On Wednesday, March 6, 2013, Jeffrey Zhong wrote:
>
> > Hey,
> >
> > Recently I'm working on some hbase test case failures and I think it's
> > useful if we can see a report of all failed test cases from most recent
> > runs. So we can easily see how flaky a test case is. I wrote a tool some
> > time back and below are some reports against different branches from
> > today's
> > run. You can get the tools' source at
> > https://github.com/jeffreyz88/jenkins-tools. If we can run it daily and
> > send
> > out an email then we can quickly notice what may break after recent
> > check-ins.
> >
> > Notes:
> > 1) " 873  874  875  876  877  878  879  880  881" are builds which has
> > failed test cases in current or previous runs
> > 2) 1 means "PASSED", 0 means "NOT RUN AT ALL", -1 means "FAILED"
> >
> > HBase-0.95 (from last 10 runs configurable)
> >
> > Failed Test Cases                21   22   23   24   25   27
> > =======================================================> >
> >
> org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h
> > adoop.hbase.catalog.testmetamigrationconvertingtopb    0    0    0    0
> > -1
> > -1
> >
> >
> org.apache.hadoop.hbase.coprocessor.example.testbulkdeleteprotocol.testbulkd
> > eletecolumn   -1    0    0    0    0    0
> >
> >
> org.apache.hadoop.hbase.coprocessor.example.testrowcountendpoint.org.apache.
> > hadoop.hbase.coprocessor.example.testrowcountendpoint   -1    0    0    0
> > 0    0
> >
> >
> org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver.
> >
> org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver
> > -1    0    0    0    0    0
> >
> >
> org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock
> > edritondeadrs    1   -1   -1    0    1    1
> >
> >
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshoul
> >
> dthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback
> > 1    1    1    1    1   -1
> >
> >
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd
> > ownfixupwhendaughterhassplit    1    1    1   -1   -1   -1
> > org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning    0
>  1
> > 1    1   -1    0
> >
> >
> org.apache.hadoop.hbase.replication.testmasterreplication.testcyclicreplicat
> > ion    1    1    1    1    1   -1
> >
> >
> org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailov
> > er    1   -1    0    1   -1    0
> >
> >
> org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.q
> > ueuefailover    0    1   -1    0   -1    0
> >
> >
> org.apache.hadoop.hbase.security.access.testaccesscontroller.org.apache.hado
> > op.hbase.security.access.testaccesscontroller    0   -1    0    0    0
>  0
> >
> > As you can see we have a few test cases don't run successfully at all or
> > recently.
> >
> > HBase-TRUNK (from last 10 runs)
> >
> > Failed Test Cases              3908 3909 3910 3912 3913 3914 3915 3916
> > =======================================================> >
> >
> org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h
> > adoop.hbase.catalog.testmetamigrationconvertingtopb    0   -1    0    0
> >  0
> > 0   -1   -1
> >
> >
> org.apache.hadoop.hbase.client.testadmin.testcloseregionwhenservernameisempt
> > y    1    1    1    1    1    1   -1    0
> > org.apache.hadoop.hbase.client.testscannertimeout.test3686a    1    1
> -1
> > 0    1    1    1    1
> >
> >
> org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testofflinesnap
> > shotregionoperationsindependent    0    1   -1    0    1    1    1    1
> >
> >
> org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testonlinesnaps
> > hotregionoperationsindependent    1    1   -1    0    1    1    1    1