Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Re: [jira] [Resolved] (HBASE-8591) Doc Improvement: Replication blog


Copy link to this message
-
Re: [jira] [Resolved] (HBASE-8591) Doc Improvement: Replication blog
Hi Himanshu,

I have a follow up question on these metrics. I see that both these metrics
basically diff against the timestamp of the WALEdit. The timestamp on the
WALEdit is generated by the region server in source cluster. So if there is
some clock skew (even 1 second) on the slave cluster, the
ageOfLastAppliedEdit could be somewhat off. On the other hand, I see that
ageOfLastShippedEdit would use the same clock (unless we have region server
failures) for calculating the time on the WALEdit and the time by which it
has been shipped. Also since the shipping is synchronous, we are sure that
the WALEdit has truly been applied on the sink cluster.

Is that correct ?

Varun
On Wed, May 22, 2013 at 11:55 AM, Demai Ni <[EMAIL PROTECTED]> wrote:

> Himanshu,
>
> Thanks a lot. Your suggestion saved me a lot of time if have to figure it
> out myself
>
> Demai on the run
>
> On May 22, 2013, at 11:00 AM, Himanshu Vashishtha <[EMAIL PROTECTED]>
> wrote:
>
> > Since the source metrics would not be available in case the master
> cluster
> > is down,
> > your best bet would be "sink.ageOfLastAppliedOp" at the slave cluster.
> > Its value is the timestamp of the last WALEdit that is applied to the
> > slave.
> >
> >
> > On Wed, May 22, 2013 at 10:02 AM, N Dm <[EMAIL PROTECTED]> wrote:
> >
> >> Stack and Himanshu,
> >>
> >> many thanks for putting this together.  Particularly  the info from
> >> Replication Metrics, whichs help a lot to understand the replication.
> >>
> >> A quick question on source.ageOfLastShippedOp. In the case that master
> >> cluster down, the user will like to know what the delta between master
> and
> >> slave. In another word, to find out the portion of mutation that already
> >> commit on Master, but haven't  replicated yet. Is
> source.ageOfLastShippedOp
> >> the right place to look at? will it be available even with HBASE down?
> >>
> >> Again thanks for the doc and appreciate it.
> >>
> >> Demai
> >>
> >>
> >> On Tue, May 21, 2013 at 10:39 PM, stack (JIRA) <[EMAIL PROTECTED]> wrote:
> >>
> >>>
> >>>     [
> >>>
> >>
> https://issues.apache.org/jira/browse/HBASE-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >> ]
> >>>
> >>> stack resolved HBASE-8591.
> >>> --------------------------
> >>>
> >>>       Resolution: Fixed
> >>>    Fix Version/s: 0.98.0
> >>>     Hadoop Flags: Reviewed
> >>>
> >>> Committed to trunk.  Thanks Himanshu (And Chris for review)
> >>>
> >>>> Doc Improvement: Replication blog
> >>>> ---------------------------------
> >>>>
> >>>>                Key: HBASE-8591
> >>>>                URL: https://issues.apache.org/jira/browse/HBASE-8591
> >>>>            Project: HBase
> >>>>         Issue Type: Task
> >>>>         Components: documentation, Replication
> >>>>   Affects Versions: 0.95.0
> >>>>           Reporter: Himanshu Vashishtha
> >>>>           Assignee: Himanshu Vashishtha
> >>>>            Fix For: 0.98.0
> >>>>
> >>>>        Attachments: HBase-8591.patch, HBase-8591-v1.patch
> >>>>
> >>>>
> >>>> Add a section for source level metrics and some truth about table
> batch
> >>> ops at sink.
> >>>
> >>> --
> >>> This message is automatically generated by JIRA.
> >>> If you think it was sent incorrectly, please contact your JIRA
> >>> administrators
> >>> For more information on JIRA, see:
> >> http://www.atlassian.com/software/jira
> >>>
> >>
>