Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Flaky Testcase Detector


Copy link to this message
-
Flaky Testcase Detector
Hey,

Recently I'm working on some hbase test case failures and I think it's
useful if we can see a report of all failed test cases from most recent
runs. So we can easily see how flaky a test case is. I wrote a tool some
time back and below are some reports against different branches from today's
run. You can get the tools' source at
https://github.com/jeffreyz88/jenkins-tools. If we can run it daily and send
out an email then we can quickly notice what may break after recent
check-ins.

Notes:
1) " 873  874  875  876  877  878  879  880  881" are builds which has
failed test cases in current or previous runs
2) 1 means "PASSED", 0 means "NOT RUN AT ALL", -1 means "FAILED"

HBase-0.95 (from last 10 runs configurable)

Failed Test Cases                21   22   23   24   25   27
=======================================================org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h
adoop.hbase.catalog.testmetamigrationconvertingtopb    0    0    0    0   -1
-1
org.apache.hadoop.hbase.coprocessor.example.testbulkdeleteprotocol.testbulkd
eletecolumn   -1    0    0    0    0    0
org.apache.hadoop.hbase.coprocessor.example.testrowcountendpoint.org.apache.
hadoop.hbase.coprocessor.example.testrowcountendpoint   -1    0    0    0
0    0
org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver.
org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver
-1    0    0    0    0    0
org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock
edritondeadrs    1   -1   -1    0    1    1
org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshoul
dthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback
1    1    1    1    1   -1
org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd
ownfixupwhendaughterhassplit    1    1    1   -1   -1   -1
org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning    0    1
1    1   -1    0
org.apache.hadoop.hbase.replication.testmasterreplication.testcyclicreplicat
ion    1    1    1    1    1   -1
org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailov
er    1   -1    0    1   -1    0
org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.q
ueuefailover    0    1   -1    0   -1    0
org.apache.hadoop.hbase.security.access.testaccesscontroller.org.apache.hado
op.hbase.security.access.testaccesscontroller    0   -1    0    0    0    0

As you can see we have a few test cases don't run successfully at all or
recently.

HBase-TRUNK (from last 10 runs)

Failed Test Cases              3908 3909 3910 3912 3913 3914 3915 3916
=======================================================org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h
adoop.hbase.catalog.testmetamigrationconvertingtopb    0   -1    0    0    0
0   -1   -1
org.apache.hadoop.hbase.client.testadmin.testcloseregionwhenservernameisempt
y    1    1    1    1    1    1   -1    0
org.apache.hadoop.hbase.client.testscannertimeout.test3686a    1    1   -1
0    1    1    1    1
org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testofflinesnap
shotregionoperationsindependent    0    1   -1    0    1    1    1    1
org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testonlinesnaps
hotregionoperationsindependent    1    1   -1    0    1    1    1    1
org.apache.hadoop.hbase.master.testassignmentmanageroncluster.testmoveregion
1    1   -1    0    1    1    1    1
org.apache.hadoop.hbase.master.testdistributedlogsplitting.testdelayeddelete
onfailure    1    1   -1    0    1    1    1    1
org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock
edritondeadrs    1    1    1    1    1   -1    0    1
org.apache.hadoop.hbase.master.testtablelockmanager.testdelete    1    1
-1    0    1    1    1    1
org.apache.hadoop.hbase.procedure.testzkprocedurecontrollers.testsimplezkcoh
ortmembercontroller    1    1    1    1    1    1   -1    0
org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testexist
ingznodeblockssplitandwerollback    1    1    1    1    1   -1    0    1
org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshoul
dthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback
1    1   -1    0    1    1    1    1
org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd
ownfixupwhendaughterhassplit   -1   -1   -1    0   -1    0   -1    0
org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning    0   -1
-1    0    1    1    1    1
org.apache.hadoop.hbase.replication.testmultislavereplication.testmultislave
replication    1    1   -1    0    1    1    1    1
org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailov
er    1    1    1   -1    0   -1    0    1
org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.q
ueuefailover    1    1    1   -1    0   -1    0    1
org.apache.hadoop.hbase.security.access.testaccesscontrolfilter.testqualifie
raccess   -1    0    1   -1    0    1    1    1
org.apache.hadoop.hbase.util.testhbasefsck.testdegenerateregions    1   -1
0    1    1    1    1    1
org.apache.hadoop.hbase.util.testhbasefsck.testfixassignmentsandnohdfschecki
ng    1   -1    0    1    1    1    1    1
org.apache.hadoop.hbase.util.testhbasefsck.testfixbytable    1   -1    0
1    1    1    1    1
org.apache.hadoop.hbase.util.testhbasefsck.testhbckthreadpooling    1   -1
0    1    1    1    1    1
org.apache.hadoop.hbase.util.testhbasefsck.testlingeringreferencefile    1
-1    0    1    1    1    1    1
org.apache.hadoop.hbase.util.testhbasefsck.testmissingfirstregion    1   -1
0    1    1    1    1    1
org.apache.hadoop.hbase.util.testhbasefsck.testmissinglastregion    1   -1
0    1    1    1    1    1
org.apache.hadoop.hbase.util.testhbasefsck.testnotinhdfs    1   -1    0    1
1    1    1    1
org.apache.hadoop.hbase.util.testhbasefsck.testnotinmetaordep
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB