Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Setting up NxN replication


Copy link to this message
-
Re: Setting up NxN replication
hi, Ishan,

can you please elaborate a bit about your use case?

for your example of 3 cluster, I think it should be set up as two M-S for
the same table:family
first one: A = Master, B = Slave
2nd one: B = slave, C = Master.
There is no need to setup the direct relationship between A and C

Demai
On Fri, Nov 8, 2013 at 1:14 PM, Ishan Chhabra <[EMAIL PROTECTED]>wrote:

> I want to setup NxN replication i.e. N clusters each replicating to each
> other. N is expected to be around 10.
>
> On doing some research, I realize it is possible after HBASE-7709 fix, but
> it would lead to much more data flowing in the system. eg.
>
> Lets say we have 3 clusters: A,B and C.
> A new write to A will go to B and then C, and also go to C directly via the
> direct path. This leads to unnecessary network usage and writes to WAL of
> B, that should be avoided. Now imagine this with 10 clusters, it won’t
> scale.
>
> One option is to create a minimum spanning tree joining all the clusters
> and make nodes replicate to their immediate peers in a master-master
> fashion. This is much better than NxN mesh, but still has extra network and
> WAL usage. It also suffers from a failure scenarios where the a single
> cluster going down will pause replication to clusters downstream.
>
> What I really want is that the ReplicationSource should only forward
> WALEdits with cluster-id same as the local cluster-id. This seems like a
> straight forward patch to put in.
>
> Any thoughts on the suggested approach or alternatives?
>
> --
> *Ishan Chhabra *| Rocket Scientist | RocketFuel Inc.
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB