Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Flaky Testcase Detector


Copy link to this message
-
Re: Flaky Testcase Detector
Nice.  Should we check it into ./dev-tools?
Thanks,
St.Ack
On Tue, Mar 5, 2013 at 7:12 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> This is handy. Thanks so much for sharing it.
>
> On Wednesday, March 6, 2013, Jeffrey Zhong wrote:
>
> > Hey,
> >
> > Recently I'm working on some hbase test case failures and I think it's
> > useful if we can see a report of all failed test cases from most recent
> > runs. So we can easily see how flaky a test case is. I wrote a tool some
> > time back and below are some reports against different branches from
> > today's
> > run. You can get the tools' source at
> > https://github.com/jeffreyz88/jenkins-tools. If we can run it daily and
> > send
> > out an email then we can quickly notice what may break after recent
> > check-ins.
> >
> > Notes:
> > 1) " 873  874  875  876  877  878  879  880  881" are builds which has
> > failed test cases in current or previous runs
> > 2) 1 means "PASSED", 0 means "NOT RUN AT ALL", -1 means "FAILED"
> >
> > HBase-0.95 (from last 10 runs configurable)
> >
> > Failed Test Cases                21   22   23   24   25   27
> > =======================================================> >
> >
> org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h
> > adoop.hbase.catalog.testmetamigrationconvertingtopb    0    0    0    0
> > -1
> > -1
> >
> >
> org.apache.hadoop.hbase.coprocessor.example.testbulkdeleteprotocol.testbulkd
> > eletecolumn   -1    0    0    0    0    0
> >
> >
> org.apache.hadoop.hbase.coprocessor.example.testrowcountendpoint.org.apache.
> > hadoop.hbase.coprocessor.example.testrowcountendpoint   -1    0    0    0
> > 0    0
> >
> >
> org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver.
> >
> org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver
> > -1    0    0    0    0    0
> >
> >
> org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmock
> > edritondeadrs    1   -1   -1    0    1    1
> >
> >
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshoul
> >
> dthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback
> > 1    1    1    1    1   -1
> >
> >
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutd
> > ownfixupwhendaughterhassplit    1    1    1   -1   -1   -1
> > org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning    0
>  1
> > 1    1   -1    0
> >
> >
> org.apache.hadoop.hbase.replication.testmasterreplication.testcyclicreplicat
> > ion    1    1    1    1    1   -1
> >
> >
> org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailov
> > er    1   -1    0    1   -1    0
> >
> >
> org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.q
> > ueuefailover    0    1   -1    0   -1    0
> >
> >
> org.apache.hadoop.hbase.security.access.testaccesscontroller.org.apache.hado
> > op.hbase.security.access.testaccesscontroller    0   -1    0    0    0
>  0
> >
> > As you can see we have a few test cases don't run successfully at all or
> > recently.
> >
> > HBase-TRUNK (from last 10 runs)
> >
> > Failed Test Cases              3908 3909 3910 3912 3913 3914 3915 3916
> > =======================================================> >
> >
> org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.h
> > adoop.hbase.catalog.testmetamigrationconvertingtopb    0   -1    0    0
> >  0
> > 0   -1   -1
> >
> >
> org.apache.hadoop.hbase.client.testadmin.testcloseregionwhenservernameisempt
> > y    1    1    1    1    1    1   -1    0
> > org.apache.hadoop.hbase.client.testscannertimeout.test3686a    1    1
> -1
> > 0    1    1    1    1
> >
> >
> org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testofflinesnap
> > shotregionoperationsindependent    0    1   -1    0    1    1    1    1
> >
> >
> org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testonlinesnaps
> > hotregionoperationsindependent    1    1   -1    0    1    1    1    1
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB