Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> how config multi regionserver, or what is wrong?


Copy link to this message
-
Re: 答复: how config multi regionserver, or what is wrong?
Your nodes are too much out of sync with time.
> Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
You need to set up time synchronization service for your cluster.

Thank you!

Sincerely,
Leonid Fedotov
Hortonworks support team
On Dec 10, 2012, at 6:17 PM, tgh wrote:

> Meanwhile , log is master , that is, blade1, is like this,  there are some ERRor like this, for  
>
> 2012-09-01 06:31:05,558 INFO org.apache.hadoop.hbase.master.ServerManager: Registering server=blade2,60020,1346452716636, regionCount=0, userLoad=false
> 2012-09-01 06:31:05,569 WARN org.apache.hadoop.hbase.master.ServerManager: Server blade4,60020,1346451768443 has been rejected; Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
> 2012-09-01 06:31:05,581 WARN org.apache.hadoop.hbase.master.ServerManager: Server blade5,60020,1346452001672 has been rejected; Reported time is too far out of sync with master.  Time difference of 263137ms > max allowed of 30000ms
> 2012-09-01 06:31:05,583 ERROR org.apache.hadoop.hbase.master.HMaster: Region server serverName=blade4,60020,1346451768443, load=(requests=0, regions=0, usedHeap=142, maxHeap=966) reported a fatal error:
> ABORTING region server serverName=blade4,60020,1346451768443, load=(requests=0, regions=0, usedHeap=142, maxHeap=966): Unhandled exception: org.apache.hadoop.hbase.ClockOutOfSyncException: Server blade4,60020,1346451768443 has been rejected; Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
> Cause:
> org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server blade4,60020,1346451768443 has been rejected; Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
>        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
>        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1574)
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.tryReportForDuty(HRegionServer.java:1531)
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:572)
>        at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server blade4,60020,1346451768443 has been rejected; Reported time is too far out of sync with master.  Time difference of 496371ms > max allowed of 30000ms
>        at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:193)
>        at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:141)
>        at org.apache.hadoop.hbase.master.HMaster.regionServerStartup(HMaster.java:675)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>        at java.lang.reflect.Method.invoke(Method.java:601)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
>        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
>
>        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>        at $Proxy5.regionServerStartup(Unknown Source)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB