Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - [Question: Replication] how to detect whether there is a replication pending?


Copy link to this message
-
Re: [Question: Replication] how to detect whether there is a replication pending?
Demai Ni 2013-09-19, 00:52
J-D and all,

sorry for coming back on the same question. I must miss something simple on
the implementation. For some reason, I always get zero for the rates

I injected the following code (in *bold*) into ReplicationSource
        LOG.debug("Replicated in total: " + this.totalReplicatedEdits);
        *LOG.debug("shippedOpsRate: " +
this.metrics.shippedOpsRate.getPreviousIntervalValue());*
       * LOG.debug("shippedBatchesRate: "
            + this.metrics.shippedBatchesRate.getPreviousIntervalValue());*

and then, run some replication UTs with debug on. The log shows that
although something is being replicated, the rate is always 0

2013-09-18 16:54:37,175 DEBUG
[RegionServer:0;localhost.localdomain,36596,1379548436390-EventThread.replicationSource,1]
regionserver.ReplicationSource(729): Replicated in total: 4
2013-09-18 16:54:37,175 DEBUG
[RegionServer:0;localhost.localdomain,36596,1379548436390-EventThread.replicationSource,1]
regionserver.ReplicationSource(730): shippedOpsRate: 0.0
2013-09-18 16:54:37,175 DEBUG
[RegionServer:0;localhost.localdomain,36596,1379548436390-EventThread.replicationSource,1]
regionserver.ReplicationSource(731): shippedBatchesRate: 0.0

Suggestions? Thanks

Demai
On Wed, Sep 18, 2013 at 10:38 AM, Jean-Daniel Cryans <[EMAIL PROTECTED]>wrote:

> Yeah, with HLog queue you can be a whole 64MB in the worst case :)
>
> Pretty hard to say for a single table what the status is, but if it was
> cluster-wide you'd just have to look at the shipped ops/batch rates (if it
> sits at 0, you're not replicating, unless there's some problem preventing
> replication).
>
> J-D
>
>
> On Wed, Sep 18, 2013 at 9:33 AM, Demai Ni <[EMAIL PROTECTED]> wrote:
>
> > hi, folks,
> >
> > I am looking for a way to flag that NO replication is pending. For
> example
> > of a simple Master-Slave setup for table t1. At a given point of time,
> can
> > I tell whether there are still Edits pending for replication? I am
> thinking
> > about using replicationsource.metrics.sizeOfLogQueue. The only concern is
> > that even with metrics.sizeOfLogQueue=0, there may still has one Log
> being
> > processed for replication. Is my understanding correct?
> >
> > Thanks
> >
> > Demai
> >
>