Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> interesting paper on log replication


Copy link to this message
-
Re: interesting paper on log replication
Thanks for the link. This paper provides an alternative, but similar
implementation to that in Zookeeper. The key difference seems to be that
the former supports membership reconfiguration.

Kafka replication is simpler because it separates the leader election part
from log replication. Such separation has a few benefits: (1) the leader
election part is easier to implement by leveraging a consensus system (e.g.
Zookeeper); (2) the log format is simpler since the log itself is not used
for leader election; (3) the replication factor for the log is decoupled
from the number of parties required for leader election (e.g., in Kafka we
can choose a replication factor of 2 for the log while using an ensemble of
5 for a Zookeeper cluster).

Both Rafe and Zookeeper are solving a harder problem than Kafka replication
because they have no consensus service to rely upon for their own leader
election since they are implementing a consensus service.

Thanks,

Jun
On Tue, Apr 9, 2013 at 10:34 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:

> Very similar in design to kafka replication
> https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf
>
> -Jay
>