Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: failover strategy


Copy link to this message
-
Re: failover strategy
The second method (replication across DCs) is not recommended.
The first set up would work provided the set of topics you are
mirroring from A->B is disjoint from the set of topics you are
mirroring from B->A (i.e., to avoid a mirroring loop).

Joel

On Fri, Jun 28, 2013 at 5:29 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I can think of two failover strategies. I am not sure which one is the right way to go.
>
> First method. set up kafka server A on cluster 1 and set up another server B on cluster 2.
> The two clusters are in different data centers. Use customized mirrormaker to sync between
> the two servers. Use one server in production and use the other one as contingency. If
> server A is down, server B will be used (this can be transparent to publishers/consumers).
> There may be a lag between the two servers before server A is down . But after A is back,
> the customized mirrormaker can sync the two. And eventually B will have all the data A had
> before the failure.
>
> Second method. Set up one kafka server using cluster 1 and cluster 2. When creating a topic ,
> always use two replications. For each partition, assign one replication to a broker in cluster 1
> and assign the other replication to a broker in cluster 2. So kafka will handle the syncing and failover
> for the two clusters. Is that a right (expected) way to use kafka?
>
>
> Regards,
>
> Libo
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB