Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - [question about replication] how to apply delta from Master to Slave after crash ?


Copy link to this message
-
[question about replication] how to apply delta from Master to Slave after crash ?
Demai Ni 2013-07-22, 23:42
hi, folks,

I am trying to address a recovery scenario after Master crash.

In a simple Master - Slave replication setup for t1, and suddenly Master
crashes. there will be a small delta of t1 already changed on Master, but
haven't been replicated to Slave yet.  Assuming Master's filesystem and
network still available, how to apply the delta from Master to Slave to
sync t1.

Anyone addresses this scenario in your shop? Can you please give me some
pointers?

Basically, I need to address two things:
1) get the timestamp of last consistent point of T1. some thing like
oldest(lastAppliedOP_of_T1 of each regionServer on Slave Cluster).  Any
other way to do so?

2) something like copyTable utility with starttime from 1). However
copyTable requests hbase and zookeep available from source cluster(right?)
I have been thinking about ways to address this for quite a few days and
still don't have very good ideas yet. Any suggestion is greatly appreciated

thanks.

Demai