Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # dev - [jira] [Updated] (KAFKA-791) Fix validation bugs in System Test


+
John Fung 2013-03-06, 00:48
+
John Fung 2013-03-06, 17:30
+
John Fung 2013-03-08, 17:00
+
John Fung 2013-03-12, 17:33
+
John Fung 2013-03-13, 20:42
Copy link to this message
-
[jira] [Updated] (KAFKA-791) Fix validation bugs in System Test
"John Fung 2013-03-18, 16:40

     [ https://issues.apache.org/jira/browse/KAFKA-791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

John Fung updated KAFKA-791:
----------------------------

    Attachment: kafka-791-v3.patch
    
> Fix validation bugs in System Test
> ----------------------------------
>
>                 Key: KAFKA-791
>                 URL: https://issues.apache.org/jira/browse/KAFKA-791
>             Project: Kafka
>          Issue Type: Task
>            Reporter: John Fung
>            Assignee: John Fung
>              Labels: replication-testing
>         Attachments: kafka-791-v1.patch, kafka-791-v2.patch, kafka-791-v3.patch
>
>
> The following issues are found in data / log checksum match in System Test:
> 1. kafka_system_test_utils.validate_simple_consumer_data_matched
> It reports PASSED even some log segments don't match
> 2. kafka_system_test_utils.validate_data_matched (this is fixed and patched in local Hudson for some time)
> It reports PASSED in the Ack=1 cases even data loss is greater than the tolerance (1%).
> 3. kafka_system_test_utils.validate_simple_consumer_data_matched
> It gets a unique set of MessageID to validate. It should leave all MessageID as is (no dedup needed) and the test case should fail if sorted MessageID don't match across the replicas.
> 4. There is a data loss tolerance of 1% in the test cases of Ack=1. Currently 1% is too strict and seeing some random failures due to 2 ~ 3% of data loss. It will be increased to 5% such that the System Test will get a more consistent passing rate in those test cases. The following will be updated to 5% tolerance in kafka_system_test_utils:
> validate_data_matched
> validate_simple_consumer_data_matched
> validate_data_matched_in_multi_topics_from_single_consumer_producer

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 
+
Jun Rao 2013-03-25, 21:09
+
John Fung 2013-03-25, 18:13