Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Differences between ScannerTimeoutException and LeaseException during a scanner


+
Guillaume Perrot 2012-11-15, 14:46
+
Guillaume Perrot 2012-11-15, 15:45
+
Ted Yu 2012-11-15, 17:43
Copy link to this message
-
Re: Differences between ScannerTimeoutException and LeaseException during a scanner
No, but we'll try that soon.

2012/11/15 Ted Yu <[EMAIL PROTECTED]>

> Thanks for sharing your experience.
>
> Have you set hbase.rpc.timeout > hbase.regionserver.lease.period in your
> environment ?
>
> Cheers
>
> On Thu, Nov 15, 2012 at 7:45 AM, Guillaume Perrot <[EMAIL PROTECTED]
> >wrote:
>
> > It looks like a "race condition" between the check that the lease is
> > expired in HRegionServer.next, and the LeaseListener.leaseExpired
> methods.
> > If the next call from the client is done before the lease listener do its
> > job, the scanner still exists in the map but is expired, which causes
> > LeaseException instead of UnknownScannerException.
> > Indeed I found this in the logs:
> >
> > 2012-11-15 08:45:34,003 ERROR
> > org.apache.hadoop.hbase.regionserver.HRegionServer:
> > org.apache.hadoop.hbase.regionserver.LeaseException: lease
> > '4939065839430667309' does not exist
> >  at
> > org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:231)
> > at
> >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2117)
> >  at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >  at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> >
> >
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> >  at
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> >
> > Then
> >
> > 2012-11-15 08:48:37,734 INFO
> > org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> > 4939065839430667309 lease expired
> >
> > The scanner is properly closed way too late!
> > I just realized there is already a related issue about this:
> > https://issues.apache.org/jira/browse/HBASE-6856
> >
> > 2012/11/15 Guillaume Perrot <[EMAIL PROTECTED]>
> >
> > > Hi,
> > > Similar questions have been mailed in the past, but they are not clear
> > > enough to me, so here is a new one:
> > > In HBase 0.90.3, when a client scanner was too slow to advance (no
> actual
> > > next call within 60s), we had a nice ScannerTimeoutException in client
> > > (UnknownScannerException in region server).
> > > With HBase 0.92.1, when I simulate such a slow scanner I still have
> this
> > > exception.
> > >
> > > But in our production environment, scanners that used to
> > > produce ScannerTimeoutException in HBase 0.90.3 no longer produce them
> > > since we upgraded to HBase 0.92.1.
> > > Instead we now have:
> > > org.apache.hadoop.hbase.regionserver.LeaseException:
> > > org.apache.hadoop.hbase.regionserver.LeaseException: lease
> > > '2770824890045864909' does not exist
> > > at
> > org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:231)
> > >  at
> > >
> >
> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:2117)
> > > at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> > >  at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > at java.lang.reflect.Method.invoke(Method.java:597)
> > >  at
> > >
> >
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> > > at
> > >
> >
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1326)
> > >
> > > I don't uderstand the workflow leading to this exception instead of a
> > > ScannerTimeoutException, any local attempt to reproduce this problem
> > leads
> > > to a ScannerTimeoutException and not a LeaseException.
> > > Any idea ?
> > > Thanks in advance.
> > >
> >
> >
> >
> > --
> >   *Guillaume Perrot
> > **Software Engineer
> > *[image: logo]
> >    *Contact info *
> > [EMAIL PROTECTED]
> > www.capptain.com
> >
> >  *Rennes Office*
> > Office: +33 2 99 65 69 13
> > 10 rue Jean-Marie Duhamel
> > 35000 Rennes
> > France *Paris Office*
> > Office: +33 1 84 06 13 85
> > Fax: +33 9 57 72 20 18
> > 18 rue Tronchet
> > 75008 Paris
> > France
> >
>
+
killaron 2013-02-12, 23:26
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB