Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Flaky Testcase Detector


Copy link to this message
-
Re: Flaky Testcase Detector
Jeffrey Zhong 2013-03-06, 18:11

I can check it into ./dev-tools unless someone objects.

Thanks,
-Jeffrey

On 3/5/13 10:43 PM, "Stack" <[EMAIL PROTECTED]> wrote:

>Nice.  Should we check it into ./dev-tools?
>Thanks,
>St.Ack
>
>
>On Tue, Mar 5, 2013 at 7:12 PM, Andrew Purtell <[EMAIL PROTECTED]>
>wrote:
>
>> This is handy. Thanks so much for sharing it.
>>
>> On Wednesday, March 6, 2013, Jeffrey Zhong wrote:
>>
>> > Hey,
>> >
>> > Recently I'm working on some hbase test case failures and I think it's
>> > useful if we can see a report of all failed test cases from most
>>recent
>> > runs. So we can easily see how flaky a test case is. I wrote a tool
>>some
>> > time back and below are some reports against different branches from
>> > today's
>> > run. You can get the tools' source at
>> > https://github.com/jeffreyz88/jenkins-tools. If we can run it daily
>>and
>> > send
>> > out an email then we can quickly notice what may break after recent
>> > check-ins.
>> >
>> > Notes:
>> > 1) " 873  874  875  876  877  878  879  880  881" are builds which has
>> > failed test cases in current or previous runs
>> > 2) 1 means "PASSED", 0 means "NOT RUN AT ALL", -1 means "FAILED"
>> >
>> > HBase-0.95 (from last 10 runs configurable)
>> >
>> > Failed Test Cases                21   22   23   24   25   27
>> > =======================================================>> >
>> >
>>
>>org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apach
>>e.h
>> > adoop.hbase.catalog.testmetamigrationconvertingtopb    0    0    0
>>0
>> > -1
>> > -1
>> >
>> >
>>
>>org.apache.hadoop.hbase.coprocessor.example.testbulkdeleteprotocol.testbu
>>lkd
>> > eletecolumn   -1    0    0    0    0    0
>> >
>> >
>>
>>org.apache.hadoop.hbase.coprocessor.example.testrowcountendpoint.org.apac
>>he.
>> > hadoop.hbase.coprocessor.example.testrowcountendpoint   -1    0    0
>>  0
>> > 0    0
>> >
>> >
>>
>>org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserv
>>er.
>> >
>>
>>org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserv
>>er
>> > -1    0    0    0    0    0
>> >
>> >
>>
>>org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithm
>>ock
>> > edritondeadrs    1   -1   -1    0    1    1
>> >
>> >
>>
>>org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testsh
>>oul
>> >
>>
>>dthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollba
>>ck
>> > 1    1    1    1    1   -1
>> >
>> >
>>
>>org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testsh
>>utd
>> > ownfixupwhendaughterhassplit    1    1    1   -1   -1   -1
>> > org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning    0
>>  1
>> > 1    1   -1    0
>> >
>> >
>>
>>org.apache.hadoop.hbase.replication.testmasterreplication.testcyclicrepli
>>cat
>> > ion    1    1    1    1    1   -1
>> >
>> >
>>
>>org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefai
>>lov
>> > er    1   -1    0    1   -1    0
>> >
>> >
>>
>>org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompresse
>>d.q
>> > ueuefailover    0    1   -1    0   -1    0
>> >
>> >
>>
>>org.apache.hadoop.hbase.security.access.testaccesscontroller.org.apache.h
>>ado
>> > op.hbase.security.access.testaccesscontroller    0   -1    0    0    0
>>  0
>> >
>> > As you can see we have a few test cases don't run successfully at all
>>or
>> > recently.
>> >
>> > HBase-TRUNK (from last 10 runs)
>> >
>> > Failed Test Cases              3908 3909 3910 3912 3913 3914 3915 3916
>> > =======================================================>> >
>> >
>>
>>org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apach
>>e.h
>> > adoop.hbase.catalog.testmetamigrationconvertingtopb    0   -1    0
>>0
>> >  0
>> > 0   -1   -1
>> >
>> >
>>
>>org.apache.hadoop.hbase.client.testadmin.testcloseregionwhenservernameise
>>mpt
>> > y    1    1    1    1    1    1   -1    0
>> > org.apache.hadoop.hbase.client.testscannertimeout.test3686a    1    1
>> -1