Looking through the HRegion.RegionScannerImpl, I see various synchronized next(...) methods, same for StoreScanner.
Scanners are created for Get operations and these scanners are guaranteed to be only used from a single thread, so in that case all synchronization is pointless.
The client's next(...) operation is a bit more interesting.

Is anybody using scanners concurrently, such as calling next(...) on the *same* scanner from multiple threads concurrently?

If not, we could enforce this once at the RegionServer level and have the entire path down to HFileScannerVx unsynchronized.

Would this work? Any counterexamples?

A quick test shows that we could gain about 10-20% during a scan when everything is in cache.

-- Lars

Alas... Store.notifyChangedReadersObservers, which calls updateReaders on all used StoreScanners, foils my ingenious plan.

________________________________
 From: lars hofhansl <[EMAIL PROTECTED]>
To: hbase-dev <[EMAIL PROTECTED]>
Sent: Saturday, December 8, 2012 7:20 PM
Subject: Concurrent use of RegionScanner.next
 
Looking through the HRegion.RegionScannerImpl, I see various synchronized next(...) methods, same for StoreScanner.
Scanners are created for Get operations and these scanners are guaranteed to be only used from a single thread, so in that case all synchronization is pointless.
The client's next(...) operation is a bit more interesting.

Is anybody using scanners concurrently, such as calling next(...) on the *same* scanner from multiple threads concurrently?

If not, we could enforce this once at the RegionServer level and have the entire path down to HFileScannerVx unsynchronized.

Would this work? Any counterexamples?

A quick test shows that we could gain about 10-20% during a scan when everything is in cache.

-- Lars

On Sat, Dec 8, 2012 at 9:32 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Alas... Store.notifyChangedReadersObservers, which calls updateReaders on
> all used StoreScanners, foils my ingenious plan.
>
>
This does not seem like enough of a reason to keep synchronization on
nexting.  Could we let readers live on across a next invocation?
St.Ack

You are right.

The scanner could potentially scan over a snapshot of all Stores at the time when the scanner started.

The details are bit tricky, though. We'd have to not compact anything while a scanner is executing (or at least keep the old files around until the scanner is scanner).
The same holds for the memstore. After the scanner started the memstore could flush, we would need to keep the state in the memstore until the scanner is done.
(TestAtomicOperation fails when the synchronization is removed from StoreScanner.next(...))
-- Lars

________________________________
 From: Stack <[EMAIL PROTECTED]>
To: HBase Dev List <[EMAIL PROTECTED]>; lars hofhansl <[EMAIL PROTECTED]>
Sent: Monday, December 10, 2012 11:17 AM
Subject: Re: Concurrent use of RegionScanner.next
 
On Sat, Dec 8, 2012 at 9:32 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Alas... Store.notifyChangedReadersObservers, which calls updateReaders on
> all used StoreScanners, foils my ingenious plan.
>
>
This does not seem like enough of a reason to keep synchronization on
nexting.  Could we let readers live on across a next invocation?
St.Ack