Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Concurrent use of RegionScanner.next


Copy link to this message
-
Re: Concurrent use of RegionScanner.next
Alas... Store.notifyChangedReadersObservers, which calls updateReaders on all used StoreScanners, foils my ingenious plan.

________________________________
 From: lars hofhansl <[EMAIL PROTECTED]>
To: hbase-dev <[EMAIL PROTECTED]>
Sent: Saturday, December 8, 2012 7:20 PM
Subject: Concurrent use of RegionScanner.next
 
Looking through the HRegion.RegionScannerImpl, I see various synchronized next(...) methods, same for StoreScanner.
Scanners are created for Get operations and these scanners are guaranteed to be only used from a single thread, so in that case all synchronization is pointless.
The client's next(...) operation is a bit more interesting.

Is anybody using scanners concurrently, such as calling next(...) on the *same* scanner from multiple threads concurrently?

If not, we could enforce this once at the RegionServer level and have the entire path down to HFileScannerVx unsynchronized.

Would this work? Any counterexamples?

A quick test shows that we could gain about 10-20% during a scan when everything is in cache.

-- Lars
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB