Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> how config multi regionserver, or what is wrong?


Copy link to this message
-
Re: 答复: how config multi regionserver, or what is wrong?
Your nodes are too much out of sync with time.
> Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
You need to set up time synchronization service for your cluster.

Thank you!

Sincerely,
Leonid Fedotov
Hortonworks support team
On Dec 10, 2012, at 6:17 PM, tgh wrote:

> Meanwhile , log is master , that is, blade1, is like this,  there are some ERRor like this, for  
>
> 2012-09-01 06:31:05,558 INFO org.apache.hadoop.hbase.master.ServerManager: Registering server=blade2,60020,1346452716636, regionCount=0, userLoad=false
> 2012-09-01 06:31:05,569 WARN org.apache.hadoop.hbase.master.ServerManager: Server blade4,60020,1346451768443 has been rejected; Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
> 2012-09-01 06:31:05,581 WARN org.apache.hadoop.hbase.master.ServerManager: Server blade5,60020,1346452001672 has been rejected; Reported time is too far out of sync with master.  Time difference of 263137ms > max allowed of 30000ms
> 2012-09-01 06:31:05,583 ERROR org.apache.hadoop.hbase.master.HMaster: Region server serverName=blade4,60020,1346451768443, load=(requests=0, regions=0, usedHeap=142, maxHeap=966) reported a fatal error:
> ABORTING region server serverName=blade4,60020,1346451768443, load=(requests=0, regions=0, usedHeap=142, maxHeap=966): Unhandled exception: org.apache.hadoop.hbase.ClockOutOfSyncException: Server blade4,60020,1346451768443 has been rejected; Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
> Cause:
> org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server blade4,60020,1346451768443 has been rejected; Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
>        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
>        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1574)
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.tryReportForDuty(HRegionServer.java:1531)
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:572)
>        at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server blade4,60020,1346451768443 has been rejected; Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
>        at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:193)
>        at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:141)
>        at org.apache.hadoop.hbase.master.HMaster.regionServerStartup(HMaster.java:675)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>        at java.lang.reflect.Method.invoke(Method.java:601)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
>        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
>
>        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>        at $Proxy5.regionServerStartup(Unknown Source)