Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - skipKVsNewerThanReadpoint in StoreFileScanner and MemStoreScanner


Copy link to this message
-
Re: skipKVsNewerThanReadpoint in StoreFileScanner and MemStoreScanner
Vladimir Rodionov 2013-10-11, 21:36
Yes, threadLocal.get() is expensive when you count every nanosecond. There
is no need to call it every time in MemStoreScanner.next().
On Fri, Oct 11, 2013 at 2:32 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> MultiVersionConsistencyControl.getThreadReadPoint() retrieves value from a
> ThreadLocal.
>
> Can you specify which operation becomes expensive ?
>
> BTW I only found IsolationLevel.READ_UNCOMMITTED
> in TestRowProcessorEndpoint. Looks like it is not normally used.
>
>
> On Fri, Oct 11, 2013 at 2:20 PM, Vladimir Rodionov
> <[EMAIL PROTECTED]>wrote:
>
> > Update.
> >
> > This method is not called in MemStoreScanner, but direct call to :
> >
> > readPoint = MultiVersionConsistencyControl.getThreadReadPoint();
> >
> > on every next, seek etc
> > which is basically the same.
> >
> > All these calls become very expensive when you try to achieve 10s of
> > millions RPS in scan operations.
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: [EMAIL PROTECTED]
> >
> > ________________________________________
> > From: Vladimir Rodionov
> > Sent: Friday, October 11, 2013 2:18 PM
> > To: [EMAIL PROTECTED]
> > Subject: skipKVsNewerThanReadpoint in StoreFileScanner and
> MemStoreScanner
> >
> > From HRegion:
> >
> > synchronized(scannerReadPoints) {
> >         if (isolationLevel == IsolationLevel.READ_UNCOMMITTED) {
> >           // This scan can read even uncommitted transactions
> >           this.readPt = Long.MAX_VALUE;
> >           MultiVersionConsistencyControl.setThreadReadPoint(this.readPt);
> >         } else {
> >           this.readPt > > MultiVersionConsistencyControl.resetThreadReadPoint(mvcc);
> >         }
> >
> > If isolation level is READ_UNCOMMITTED than this method should never be
> > called in both StoreFileScanner and MemStoreScanner
> > If isolation level is READ_COMMITTED than this method must be called only
> > in MemStoreScanner, because HFiles are immutable no new KV's can be added
> > to HFile after StoreFileScanner was created.
> >
> >
> > Correct?
> >
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: [EMAIL PROTECTED]
> >
> > Confidentiality Notice:  The information contained in this message,
> > including any attachments hereto, may be confidential and is intended to
> be
> > read only by the individual or entity to whom this message is addressed.
> If
> > the reader of this message is not the intended recipient or an agent or
> > designee of the intended recipient, please note that any review, use,
> > disclosure or distribution of this message or its attachments, in any
> form,
> > is strictly prohibited.  If you have received this message in error,
> please
> > immediately notify the sender and/or [EMAIL PROTECTED] and
> > delete or destroy any copy of this message and its attachments.
> >
>