Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Replication across Multiple Datacenters

Copy link to this message
Replication across Multiple Datacenters
Mark Farnan 2013-06-24, 01:11

Is the replication factor system in  Kafka 0.8 suitable for creating single clusters which span across data centers ?  (up to 3)

I am looking for a system where I don't loose messages, and can effectively 'fail over' to a different datacenter for processing if/when the primary goes down.   If I read correctly,  any message delivered to a Kafka broker, will be copied to its Replica/s.  And a producer could deliver messages to any broker in the same replica set.  

Is that correct ?
I am aware there are several zookeeper issues around multi DC support which I need to sort out,   so this question is specific for the Kafka portion.

Note: My main consumer from Kafka will be STORM.