lars hofhansl 2012-12-09, 03:20
-Re: Concurrent use of RegionScanner.next
lars hofhansl 2012-12-09, 05:32
Alas... Store.notifyChangedReadersObservers, which calls updateReaders on all used StoreScanners, foils my ingenious plan.
From: lars hofhansl <[EMAIL PROTECTED]>
To: hbase-dev <[EMAIL PROTECTED]>
Sent: Saturday, December 8, 2012 7:20 PM
Subject: Concurrent use of RegionScanner.next
Looking through the HRegion.RegionScannerImpl, I see various synchronized next(...) methods, same for StoreScanner.
Scanners are created for Get operations and these scanners are guaranteed to be only used from a single thread, so in that case all synchronization is pointless.
The client's next(...) operation is a bit more interesting.
Is anybody using scanners concurrently, such as calling next(...) on the *same* scanner from multiple threads concurrently?
If not, we could enforce this once at the RegionServer level and have the entire path down to HFileScannerVx unsynchronized.
Would this work? Any counterexamples?
A quick test shows that we could gain about 10-20% during a scan when everything is in cache.
Stack 2012-12-10, 19:17
lars hofhansl 2012-12-11, 05:01