Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Getting the scan type at preCompact


+
Mesika, Asaf 2013-01-29, 13:23
+
ramkrishna vasudevan 2013-01-29, 14:45
+
Asaf Mesika 2013-01-29, 18:21
+
Ted Yu 2013-01-29, 18:33
+
lars hofhansl 2013-01-29, 18:55
+
Ted Yu 2013-01-29, 19:29
Copy link to this message
-
Re: Getting the scan type at preCompact
lars hofhansl 2013-01-29, 19:58
Oh yeah, wasn't disagreeing with that, just that until we have a patch there's a workaround.

________________________________
 From: Ted Yu <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
Sent: Tuesday, January 29, 2013 11:29 AM
Subject: Re: Getting the scan type at preCompact
 

Since preCompact() is given the scanner returned from preCompactScannerOpen(), I think for users it is more convenient to wrap this scanner in preCompact().

Cheers
On Tue, Jan 29, 2013 at 10:55 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

I added the preCompactScannerOpen() to RegionObserver and didn't go back and also changes preCompact.
>In lieu of a fix you could create the scanner in preCompactScannerOpen() (take a look at Compactor.compact() to see how the scanner would be created).
>
>-- Lars
>
>
>
>________________________________
> From: "Mesika, Asaf" <[EMAIL PROTECTED]>
>To: [EMAIL PROTECTED]
>Sent: Tuesday, January 29, 2013 5:23 AM
>Subject: Getting the scan type at preCompact
>
>
>Hi,
>
>In the RegionObserver.preCompactScannerOpen() method, one of the parameters is scanType which enables me to know if the compaction is major or minor.
>In the preCompact() method I don't have that parameter.
>
>In a region observer I wrote, I'm basically wrapping the InternalScanner with my own Scanner.
>My scanner should behave differently when its a a major or minor compaction.
>
>Due to this restriction I'm forced to use preCompactScannerOpen() and create the StoreScanner, copy pasting the code that creates it in Store.compactStore() (marked in green):
>
>        if (getHRegion().getCoprocessorHost() != null) {
>          scanner = getHRegion()
>              .getCoprocessorHost()
>              .preCompactScannerOpen(this, scanners,
>                  majorCompaction ? ScanType.MAJOR_COMPACT : ScanType.MINOR_COMPACT, earliestPutTs);
>        }
>        if (scanner == null) {
>          Scan scan = new Scan();
>          scan.setMaxVersions(getFamily().getMaxVersions());
>          /* Include deletes, unless we are doing a major compaction */
>          scanner = new StoreScanner(this, getScanInfo(), scan, scanners,
>            majorCompaction? ScanType.MAJOR_COMPACT : ScanType.MINOR_COMPACT,
>            smallestReadPoint, earliestPutTs);
>        }
>        if (getHRegion().getCoprocessorHost() != null) {
>          InternalScanner cpScanner >            getHRegion().getCoprocessorHost().preCompact(this, scanner);
>          // NULL scanner returned from coprocessor hooks means skip normal processing
>          if (cpScanner == null) {
>            return null;
>          }
>          scanner = cpScanner;
>        }
>
>
>Can I file a JIRA to add the Scan Type to the preCompact method?
>
>Thanks,
>
>Asaf
+
Ted Yu 2013-01-29, 20:33
+
Ted Yu 2013-01-30, 02:53
+
ramkrishna vasudevan 2013-01-30, 03:54