Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> state of replication in 0.8

Stan Rosenberg 2013-01-20, 00:38
Copy link to this message
Re: state of replication in 0.8

Quite a few people have been asking the same question. I created this wiki (
https://cwiki.apache.org/confluence/display/KAFKA/Changes+in+Kafka+0.8) to
document the significant changes in 0.8.

The replication design in 0.8 leveraged some ideas in the paper that you
mentioned, but is not the same. The key difference is that in 0.8, we
tolerate f-1 failures with f replicas. In contrast, typical Paxos-based
algorithms only tolerate f failures with 2*f + 1 replicas.

In terms of release schedule, we have seen some blocker issues when testing
0.8 on some production data at LinkedIn. Those blockers are being tracked
We plan to release a beta version once those blockers are resolved,
hopefully in the next few weeks.



On Sat, Jan 19, 2013 at 4:38 PM, Stan Rosenberg <[EMAIL PROTECTED]>wrote:

> Hi,
> Could someone please comment on the state of replication in 0.8, i.e.,
> what's implemented and what remains to be implemented?  On a first
> blush I see similarity between what's in the
> high-level documentation and Jun's paper, "Using Paxos to Build a
> Scalable, Consistent, and Highly Available Datastore".  I'd be
> especially grateful if someone could highlight the differences.
> Thanks!
> stan