Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Differences between ScannerTimeoutException and LeaseException during a scanner


+
Guillaume Perrot 2012-11-15, 14:46
+
Guillaume Perrot 2012-11-15, 15:45
Copy link to this message
-
Re: Differences between ScannerTimeoutException and LeaseException during a scanner
Thanks for sharing your experience.

Have you set hbase.rpc.timeout > hbase.regionserver.lease.period in your
environment ?

Cheers

On Thu, Nov 15, 2012 at 7:45 AM, Guillaume Perrot <[EMAIL PROTECTED]>wrote:

> It looks like a "race condition" between the check that the lease is
> expired in HRegionServer.next, and the LeaseListener.leaseExpired methods.
> If the next call from the client is done before the lease listener do its
> job, the scanner still exists in the map but is expired, which causes
> LeaseException instead of UnknownScannerException.
> Indeed I found this in the logs:
>
> 2012-11-15 08:45:34,003 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> org.apache.hadoop.hbase.regionserver.LeaseException: lease
> '4939065839430667309' does not exist
>  at
> org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:231)
> at
>
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2117)
>  at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>  at java.lang.reflect.Method.invoke(Method.java:597)
> at
>
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
>  at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
>
> Then
>
> 2012-11-15 08:48:37,734 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> 4939065839430667309 lease expired
>
> The scanner is properly closed way too late!
> I just realized there is already a related issue about this:
> https://issues.apache.org/jira/browse/HBASE-6856
>
> 2012/11/15 Guillaume Perrot <[EMAIL PROTECTED]>
>
> > Hi,
> > Similar questions have been mailed in the past, but they are not clear
> > enough to me, so here is a new one:
> > In HBase 0.90.3, when a client scanner was too slow to advance (no actual
> > next call within 60s), we had a nice ScannerTimeoutException in client
> > (UnknownScannerException in region server).
> > With HBase 0.92.1, when I simulate such a slow scanner I still have this
> > exception.
> >
> > But in our production environment, scanners that used to
> > produce ScannerTimeoutException in HBase 0.90.3 no longer produce them
> > since we upgraded to HBase 0.92.1.
> > Instead we now have:
> > org.apache.hadoop.hbase.regionserver.LeaseException:
> > org.apache.hadoop.hbase.regionserver.LeaseException: lease
> > '2770824890045864909' does not exist
> > at
> org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:231)
> >  at
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2117)
> > at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> >  at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> >  at
> >
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> > at
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> >
> > I don't uderstand the workflow leading to this exception instead of a
> > ScannerTimeoutException, any local attempt to reproduce this problem
> leads
> > to a ScannerTimeoutException and not a LeaseException.
> > Any idea ?
> > Thanks in advance.
> >
>
>
>
> --
>   *Guillaume Perrot
> **Software Engineer
> *[image: logo]
>    *Contact info *
> [EMAIL PROTECTED]
> www.capptain.com
>
>  *Rennes Office*
> Office: +33 2 99 65 69 13
> 10 rue Jean-Marie Duhamel
> 35000 Rennes
> France *Paris Office*
> Office: +33 1 84 06 13 85
> Fax: +33 9 57 72 20 18
> 18 rue Tronchet
> 75008 Paris
> France
>
> IMPORTANT NOTICE – UBIKOD and CAPPTAIN are registered trademarks of UBIKOD
> S.A.R.L., all copyrights are reserved. The contents of this email and
> attachments are confidential and may be subject to legal privilege and/or
> protected by copyright. Copying or communicating any part of it to others
> is prohibited and may be unlawful. If you are not the intended recipient
+
Guillaume Perrot 2012-11-15, 18:12
+
killaron 2013-02-12, 23:26
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB