Hi,

We closed the following jira this week. A few other jiras have patches and
are under review.
KAFKA-329: Create topic support (revisit based on v3 design)

We have people signed up on the following jiras:

KAFKA-339: Use multi-fetch for follower replication
KAFKA-350: Improve the highwatermark maintenance to handle failures
KAFKA-353: Tie Producer-side ack with high watermark and progress of
replicas
KAFKA-343: Leader election, become leader, become follower (v3. A,C)
KAFKA-338: controller failover (V3. D)
KAFKA-354: refactor setter/getter in scala style

The following jiras can be worked on in parallel and we welcome contribution
from the community.

KAFKA-330: Delete topic support
KAFKA-369: remove ZK dependency on producer
KAFKA-288: rework java producer API
KAFKA-289: reuse TopicData when sending produce request
KAFKA-355: Move the highwatermark maintenance logic out of Log.
KAFKA-356: create a generic thread class

Thanks,

Jun

Hi,

We closed the following jiras this week.
KAFKA-339: Use multi-fetch for follower replication
KAFKA-353: Tie Producer-side ack with high watermark and progress of
replicas

We have people signed up on the following jiras:
KAFKA-350: Improve the highwatermark maintenance to handle failures
KAFKA-343: Leader election, become leader, become follower (v3. A,C)
KAFKA-338: controller failover (V3. D)
KAFKA-354: refactor setter/getter in scala style

The following jiras can be worked on in parallel and we
welcome contribution from the community.
KAFKA-376: expose different data to fetch requests from the follower
replicas and consumer clients
KAFKA-386: Race condition in accessing ISR
KAFKA-385: RequestPurgatory enhancements
KAFKA-369: remove ZK dependency on producer
KAFKA-288: rework java producer API
KAFKA-289: reuse TopicData when sending produce request
KAFKA-355: Move the highwatermark maintenance logic out of Log.
KAFKA-384: Fix intermittent test failures and remove unnecessary sleeps
KAFKA-356: create a generic thread class

Thanks,

Jun

Hi,

We didn't close any jira this week, but have patch under review for the
following jiras.
KAFKA-343: Leader election, become leader, become follower (v3. A,C)
KAFKA-338: controller failover (V3. D)

We have people signed up on the following jiras:
KAFKA-350: Improve the highwatermark maintenance to handle failures
KAFKA-354: refactor setter/getter in scala style
KAFKA-362: ZKConsumerConnector needs to reconnect to new leader after
leader change in a partition

Once KAFKA-343,KAFKA-338,KAFKA-350 and KAFKA-362 are done (which we expect
to happen in the next week or two), most big jiras for phase 2 are done.
After that, we will spend some time to close some of the smaller jiras and
then proceed with phase 3.

The following jiras can be worked on in parallel and we
welcome contribution from the community.
KAFKA-376: expose different data to fetch requests from the follower
replicas and consumer clients
KAFKA-386: Race condition in accessing ISR
KAFKA-385: RequestPurgatory enhancements
KAFKA-369: remove ZK dependency on producer
KAFKA-288: rework java producer API
KAFKA-289: reuse TopicData when sending produce request
KAFKA-391: Producer request and response classes should use maps
KAFKA-355: Move the highwatermark maintenance logic out of Log.
KAFKA-384: Fix intermittent test failures and remove unnecessary sleeps
KAFKA-356: create a generic thread class

Thanks,

Jun

Hi,

We close the following jira this week.
KAFKA-362: ZKConsumerConnector needs to reconnect to new leader after
leader change in a partition

The following jiras are under review.
KAFKA-343: Leader election, become leader, become follower (v3. A,C)
KAFKA-338: controller failover (V3. D)

We have people signed up on the following jiras:
KAFKA-350: Improve the highwatermark maintenance to handle failures
KAFKA-354: refactor setter/getter in scala style
KAFKA-376: expose different data to fetch requests from the follower
replicas and consumer clients

The following jiras can be worked on in parallel and we
welcome contribution from the community.
KAFKA-386: Race condition in accessing ISR
KAFKA-385: RequestPurgatory enhancements
KAFKA-369: remove ZK dependency on producer
KAFKA-288: rework java producer API
KAFKA-289: reuse TopicData when sending produce request
KAFKA-391: Producer request and response classes should use maps
KAFKA-355: Move the highwatermark maintenance logic out of Log.
KAFKA-384: Fix intermittent test failures and remove unnecessary sleeps
KAFKA-356: create a generic thread class

Thanks,

Jun

Hi,

We closed 3 jiras this week,

KAFKA-352 Send error code to client for requests sent to non-existing topics
KAFKA-371 Refactoring of LogManager
KAFKA-362: ZKConsumerConnector needs to reconnect to new leader after
leader change in a partition

We have patches under review for the following jiras -

KAFKA-350: Enable message replication in the presence of controlled failures
KAFKA-405  Improve highwatermark maintenance to store high watermarks
in a single file
KAFKA-343: Leader election, become leader, become follower

We have people signed up on the following jiras:

KAFKA-354: refactor setter/getter in scala style
KAFKA-376: expose different data to fetch requests from the follower
replicas and consumer clients
KAFKA-386: Race condition in accessing ISR

Once KAFKA-343,KAFKA-338,KAFKA-350 are done (which we expect to happen
in the next week or two), most big jiras for phase 2 are done.
After that, we will spend time getting the system test suite in shape

The following jiras can be worked on in parallel and we welcome
contribution from the community.

KAFKA-385: RequestPurgatory enhancements
KAFKA-369: remove ZK dependency on producer
KAFKA-288: rework java producer API
KAFKA-289: reuse TopicData when sending produce request
KAFKA-391: Producer request and response classes should use maps
KAFKA-384: Fix intermittent test failures and remove unnecessary sleeps
KAFKA-356: create a generic thread class

Thanks,
Neha

Thanks for the status update.
> The following jiras can be worked on in parallel and we welcome
> contribution from the community.
>
> KAFKA-385: RequestPurgatory enhancements
>

I started on the above, although got side-tracked with some other work.
Will get back to it next week.

Joel

Hi,

We closed 5 jiras this week -

KAFKA-350: Enable message replication in the presence of controlled failures
KAFKA-405 Improve high watermark maintenance to store high watermarks
in a single file
KAFKA-412 deal with empty TopicData list in producer and fetch request
KAFKA-413 single_host_multi_brokers system test fails on laptop
KAFKA-418 NullPointerException in ConsumerFetcherManager

We have patch under review for the following jira -

KAFKA-343: Leader election, become leader, become follower

We have people signed up on the following jiras:

KAFKA-354: refactor setter/getter in scala style
KAFKA-376: expose different data to fetch requests from the follower
replicas and consumer clients
KAFKA-386: Race condition in accessing ISR
KAFKA-385: RequestPurgatory enhancements
KAFKA-369: remove ZK dependency on producer
KAFKA-384: Fix intermittent test failures and remove unnecessary sleeps

With KAFKA-350 resolved, message replication is possible in the
presence of broker side failures. We will now focus on building a
system test suite to add more performance/regression tests

The following jiras can be worked on in parallel and we welcome
contribution from the community.

KAFKA-288: rework java producer API
KAFKA-289: reuse TopicData when sending produce request
KAFKA-391: Producer request and response classes should use maps
KAFKA-356: create a generic thread class

Thanks,

Neha

Hi,

We closed 2 jiras this week, of which KAFKA-343 is the last large remaining
jira in phase 2.
KAFKA-343: Leader election, become leader, become follower
KAFKA-384: Fix intermittent test failures and remove unnecessary sleeps

We have people signed up on the following jiras:
KAFKA-354: refactor setter/getter in scala style
KAFKA-376: expose different data to fetch requests from the follower
replicas and consumer clients
KAFKA-386: Race condition in accessing ISR
KAFKA-385: RequestPurgatory enhancements
KAFKA-369: remove ZK dependency on producer
KAFKA-181: Log errors for unrecognized config options

We still have a bunch of clean up jiras left for phase 2. The following
jiras can be worked on in parallel and we welcome
contribution from the community.

KAFKA-415: Controller throws NoSuchElementException
KAFKA-416: Controller tests throw several zookeeper errors
KAFKA-420: maintain HW correctly with only 1 replica in ISR
KAFKA-422: LazyInitProducerTest has transient test failure
KAFKA-427, KAFKA-421: LogRecoverTest has transient failure
KAFKA-428: need to update leaderAndISR path in ZK conditionally
KAFKA-433: ensurePartitionLeaderOnThisBroker should not access ZK
KAFKA-434: IllegalMonitorStateException in ReplicaManager.makerFollower
KAFKA-288: rework java producer API
KAFKA-289: reuse TopicData when sending produce request
KAFKA-391: Producer request and response classes should use maps
KAFKA-356: create a generic thread class

Thanks,

Jun

Thanks to Joel Koshy, we now have a dashboard for tracking the remaining
jiras of Kafka replication. We welcome contributions from the community.

https://cwiki.apache.org/confluence/display/KAFKA/Kafka+replication+development

We closed a few jiras this week, the largest of which is
kafka-385: jmx enhancement

We plan to spend another week or two to clean up the remaining bugs and
factor code a bit.

Thanks,

Jun