Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - hbase time skew


+
Yuling_C@... 2013-03-15, 22:49
+
Ted Yu 2013-03-15, 22:53
+
Yuling_C@... 2013-03-15, 23:26
+
Ted Yu 2013-03-15, 23:28
+
Yuling_C@... 2013-03-15, 23:50
+
Ted Yu 2013-03-16, 00:07
+
Ted Yu 2013-03-16, 00:12
+
Yuling_C@... 2013-03-16, 00:24
Copy link to this message
-
Re: hbase time skew
Ted Yu 2013-03-16, 00:28
You can add them to your hbase-site.xml

Cheers

On Fri, Mar 15, 2013 at 5:24 PM, <[EMAIL PROTECTED]> wrote:

>
> Thanks very much Ted for the information. That's very helpful.
>
> Just wondering if you have any idea where to configure these two values? I
> don't see these two parameters in any files under hbase/conf?
>
> Thanks,
>
> YuLing
> -----Original Message-----
> From: Ted Yu [mailto:[EMAIL PROTECTED]]
> Sent: Friday, March 15, 2013 5:13 PM
> To: [EMAIL PROTECTED]
> Subject: Re: hbase time skew
>
> I just realized that if the clock on region server(s) drifts over
> relatively long period of time (assuming region server stays up), the
> following mechanism wouldn't work.
>
> FYI
>
> On Fri, Mar 15, 2013 at 5:07 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > When a region server reports to master the first time, ServerManager
> > does the following check:
> >
> >    * Checks if the clock skew between the server and the master. If
> > the clock skew exceeds the
> >
> >    * configured max, it will throw an exception; if it exceeds the
> > configured warning threshold,
> >
> >    * it will log a warning but start normally.
> > The unit is in milliseconds.
> > These two parameters are configurable.
> >
> > Please take a look at the source code if possible - the code should
> > come with tar ball.
> >
> > Cheers
> >
> > On Fri, Mar 15, 2013 at 4:50 PM, <[EMAIL PROTECTED]> wrote:
> >
> >>  Hi Ted,
> >>
> >> For the source code below, what's the unit of 30000? Is the maxSkew
> >> 30 seconds?  Also, what does Hbase do when the time skew exeeds 10
> >> seconds and
> >> 30 seconds? Are these two values configurable?
> >>
> >> Thanks,
> >>
> >> YuLing
> >> >maxSkew = c.getLong("hbase.master.maxclockskew", 30000);
> >> >     warningSkew = c.getLong("hbase.master.warningclockskew",
> >> >10000);
> >> >./src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
> >>
> >>
> >> -----Original Message-----
> >> From: Ted Yu [mailto:[EMAIL PROTECTED]]
> >> Sent: Friday, March 15, 2013 4:28 PM
> >> To: [EMAIL PROTECTED]
> >> Subject: Re: hbase time skew
> >>
> >> Please upgrade to 0.94.5, if possible.
> >>
> >> There have been a lot of bug fixes and performance improvements since
> >> 0.94.1.
> >>
> >> On Fri, Mar 15, 2013 at 4:26 PM, <[EMAIL PROTECTED]> wrote:
> >>
> >> > Yes, I'm using 0.94.1.
> >> >
> >> > -----Original Message-----
> >> > From: Ted Yu [mailto:[EMAIL PROTECTED]]
> >> > Sent: Friday, March 15, 2013 3:53 PM
> >> > To: [EMAIL PROTECTED]
> >> > Subject: Re: hbase time skew
> >> >
> >> > Are you using 0.94.x ?
> >> >
> >> > If so, see the following:
> >> >
> >> >     maxSkew = c.getLong("hbase.master.maxclockskew", 30000);
> >> >     warningSkew = c.getLong("hbase.master.warningclockskew",
> >> > 10000);
> >> > ./src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
> >> >
> >> > Cheers
> >> >
> >> > On Fri, Mar 15, 2013 at 3:49 PM, <[EMAIL PROTECTED]> wrote:
> >> >
> >> > > Hi,
> >> > >
> >> > > We recently encountered the issue that HBase tables got into a
> >> > > state that was not disabled nor enabled. We found that the root
> >> > > cause was the linux clock skewed more than 5 hours. I googled and
> >> > > understood that hbase can only  handle about a couple of seconds
> >> > > time skew. We were wondering if there's any configuration in
> >> > > HBase that we can do so as to increase the number of seconds that
> hbase could handle?
> >> > >
> >> > > Thanks very much,
> >> > >
> >> > > YuLing
> >> > >
> >> >
> >>
> >
> >
>
+
Michel Segel 2013-03-16, 01:56
+
Kevin Odell 2013-03-16, 12:48
+
Yuling_C@... 2013-03-18, 18:39
+
Michel Segel 2013-03-18, 21:29
+
Yuling_C@... 2013-03-18, 22:06