Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Is this exception caused by an overloaded node?


Copy link to this message
-
Is this exception caused by an overloaded node?
Hi,

I am running on a cluster where some of the machines are loaded for other
purposes. Occasionally an HBase scan fails with the message below, and I
suspect this is caused by on or more of the region servers being overloaded
with other processes (not hadoop/hbase) and not being able to respond
correctly. Is this possible?

Thanks,

Jon

Exception in thread "main" java.lang.RuntimeException:
org.apache.hadoop.hbase.regionserver.LeaseException:
org.apache.hadoop.hbase.regionserver.LeaseException: lease
'-3573171992963675348' does not exist
    at
org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:231)
    at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2117)
    at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
    at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)

    at
org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1402)
    at prd.Levelize.readPins(Levelize.java:50)
    at prd.Levelize.main(Levelize.java:245)
Caused by: org.apache.hadoop.hbase.regionserver.LeaseException:
org.apache.hadoop.hbase.regionserver.LeaseException: lease
'-3573171992963675348' does not exist
    at
org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:231)
    at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2117)
    at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
    at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
    at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
    at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:84)
    at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:39)
    at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1325)
    at
org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1293)
    at
org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1399)
    ... 2 more