Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> hbase time skew


+
Yuling_C@... 2013-03-15, 22:49
+
Ted Yu 2013-03-15, 22:53
+
Yuling_C@... 2013-03-15, 23:26
+
Ted Yu 2013-03-15, 23:28
+
Yuling_C@... 2013-03-15, 23:50
+
Ted Yu 2013-03-16, 00:07
+
Ted Yu 2013-03-16, 00:12
+
Yuling_C@... 2013-03-16, 00:24
Copy link to this message
-
Re: hbase time skew
You can add them to your hbase-site.xml

Cheers

On Fri, Mar 15, 2013 at 5:24 PM, <[EMAIL PROTECTED]> wrote:

>
> Thanks very much Ted for the information. That's very helpful.
>
> Just wondering if you have any idea where to configure these two values? I
> don't see these two parameters in any files under hbase/conf?
>
> Thanks,
>
> YuLing
> -----Original Message-----
> From: Ted Yu [mailto:[EMAIL PROTECTED]]
> Sent: Friday, March 15, 2013 5:13 PM
> To: [EMAIL PROTECTED]
> Subject: Re: hbase time skew
>
> I just realized that if the clock on region server(s) drifts over
> relatively long period of time (assuming region server stays up), the
> following mechanism wouldn't work.
>
> FYI
>
> On Fri, Mar 15, 2013 at 5:07 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > When a region server reports to master the first time, ServerManager
> > does the following check:
> >
> >    * Checks if the clock skew between the server and the master. If
> > the clock skew exceeds the
> >
> >    * configured max, it will throw an exception; if it exceeds the
> > configured warning threshold,
> >
> >    * it will log a warning but start normally.
> > The unit is in milliseconds.
> > These two parameters are configurable.
> >
> > Please take a look at the source code if possible - the code should
> > come with tar ball.
> >
> > Cheers
> >
> > On Fri, Mar 15, 2013 at 4:50 PM, <[EMAIL PROTECTED]> wrote:
> >
> >>  Hi Ted,
> >>
> >> For the source code below, what's the unit of 30000? Is the maxSkew
> >> 30 seconds?  Also, what does Hbase do when the time skew exeeds 10
> >> seconds and
> >> 30 seconds? Are these two values configurable?
> >>
> >> Thanks,
> >>
> >> YuLing
> >> >maxSkew = c.getLong("hbase.master.maxclockskew", 30000);
> >> >     warningSkew = c.getLong("hbase.master.warningclockskew",
> >> >10000);
> >> >./src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
> >>
> >>
> >> -----Original Message-----
> >> From: Ted Yu [mailto:[EMAIL PROTECTED]]
> >> Sent: Friday, March 15, 2013 4:28 PM
> >> To: [EMAIL PROTECTED]
> >> Subject: Re: hbase time skew
> >>
> >> Please upgrade to 0.94.5, if possible.
> >>
> >> There have been a lot of bug fixes and performance improvements since
> >> 0.94.1.
> >>
> >> On Fri, Mar 15, 2013 at 4:26 PM, <[EMAIL PROTECTED]> wrote:
> >>
> >> > Yes, I'm using 0.94.1.
> >> >
> >> > -----Original Message-----
> >> > From: Ted Yu [mailto:[EMAIL PROTECTED]]
> >> > Sent: Friday, March 15, 2013 3:53 PM
> >> > To: [EMAIL PROTECTED]
> >> > Subject: Re: hbase time skew
> >> >
> >> > Are you using 0.94.x ?
> >> >
> >> > If so, see the following:
> >> >
> >> >     maxSkew = c.getLong("hbase.master.maxclockskew", 30000);
> >> >     warningSkew = c.getLong("hbase.master.warningclockskew",
> >> > 10000);
> >> > ./src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
> >> >
> >> > Cheers
> >> >
> >> > On Fri, Mar 15, 2013 at 3:49 PM, <[EMAIL PROTECTED]> wrote:
> >> >
> >> > > Hi,
> >> > >
> >> > > We recently encountered the issue that HBase tables got into a
> >> > > state that was not disabled nor enabled. We found that the root
> >> > > cause was the linux clock skewed more than 5 hours. I googled and
> >> > > understood that hbase can only  handle about a couple of seconds
> >> > > time skew. We were wondering if there's any configuration in
> >> > > HBase that we can do so as to increase the number of seconds that
> hbase could handle?
> >> > >
> >> > > Thanks very much,
> >> > >
> >> > > YuLing
> >> > >
> >> >
> >>
> >
> >
>
+
Michel Segel 2013-03-16, 01:56
+
Kevin Odell 2013-03-16, 12:48
+
Yuling_C@... 2013-03-18, 18:39
+
Michel Segel 2013-03-18, 21:29
+
Yuling_C@... 2013-03-18, 22:06
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB