Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> [jira] [Resolved] (KAFKA-866) Recover segment does shallow iteration to fix index causing inconsistencies


Copy link to this message
-
[jira] [Resolved] (KAFKA-866) Recover segment does shallow iteration to fix index causing inconsistencies

     [ https://issues.apache.org/jira/browse/KAFKA-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jun Rao resolved KAFKA-866.
---------------------------

       Resolution: Fixed
    Fix Version/s: 0.8

Thanks for the patch. +1. Committed to 0.8.
                
> Recover segment does shallow iteration to fix index causing inconsistencies
> ---------------------------------------------------------------------------
>
>                 Key: KAFKA-866
>                 URL: https://issues.apache.org/jira/browse/KAFKA-866
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: Sriram Subramanian
>            Assignee: Sriram Subramanian
>            Priority: Blocker
>              Labels: kafka-0.8, p1
>             Fix For: 0.8
>
>         Attachments: KAFKA-866.patch, KAFKA-866-v2.patch
>
>
> When we recover a segment, we do a shallow iteration to fix the index. This creates entries in the index with the last logical offset in a compressed messageSet. However, during log append we use the starting offset in a compressed message set to populate the index (We changed to starting offset recently). This causes inconsistencies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB