Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Getting the scan type at preCompact


+
Mesika, Asaf 2013-01-29, 13:23
+
ramkrishna vasudevan 2013-01-29, 14:45
+
Asaf Mesika 2013-01-29, 18:21
Copy link to this message
-
Re: Getting the scan type at preCompact
Ted Yu 2013-01-29, 18:33
Asaf:
I guess you're suggesting that we pass ScanType into preCompact() similar
to what we pass into preCompactScannerOpen():

              .preCompactScannerOpen(store, scanners,

                  majorCompaction ? ScanType.MAJOR_COMPACT : ScanType.
MINOR_COMPACT, earliestPutTs);
Mind filing a JIRA ?

Thanks

On Tue, Jan 29, 2013 at 10:21 AM, Asaf Mesika <[EMAIL PROTECTED]> wrote:

> But my scanner is a wrapper to the scanner passed on to me in preCompact.
> This scanner is created in the Store object (StoreScanner) with arguments I
> don't have at preCompactScannerOpen(). So I can't create it. I want the
> vanilla scanner hbase uses is preCompact so I can wrap it with my behavior
> but also I need to know whether its minor or major.
> Currently I hacked my region observer so that in preCompactScannerOpen I
> save the ScanType in my instance as a field variable (ThreadLocal to be on
> the safe side) and in preCompact I read it and use it to build my wrapped
> scanner.
>
> Sent from my iPhone
>
> On 29 בינו 2013, at 16:46, ramkrishna vasudevan <
> [EMAIL PROTECTED]> wrote:
>
> You can do the same with preCompactScannerOpen() hook right.
> If the scanType is minor send a wrapped internalScanner say
> MinorCompactionScanner.
>
> If the scanType is major send a wrapped interanlScanner say
> MajorCompactionScanner.
>
> Does that solve your purpose? Because if preCompactScannerOpen returns a
> scanner only that will  be used inside preCompact also.
>
> Regards
> Ram
>
> On Tue, Jan 29, 2013 at 6:53 PM, Mesika, Asaf <[EMAIL PROTECTED]>
> wrote:
>
> Hi,
>
>
> In the RegionObserver.preCompactScannerOpen() method, one of the
>
> parameters is scanType which enables me to know if the compaction is major
>
> or minor.
>
> In the preCompact() method I don't have that parameter.
>
>
> In a region observer I wrote, I'm basically wrapping the InternalScanner
>
> with my own Scanner.
>
> My scanner should behave differently when its a a major or minor
>
> compaction.
>
>
> Due to this restriction I'm forced to use preCompactScannerOpen() and
>
> create the StoreScanner, copy pasting the code that creates it in
>
> Store.compactStore() (marked in green):
>
>
>        if (getHRegion().getCoprocessorHost() != null) {
>
>          scanner = getHRegion()
>
>              .getCoprocessorHost()
>
>              .preCompactScannerOpen(this, scanners,
>
>                  majorCompaction ? ScanType.MAJOR_COMPACT :
>
> ScanType.MINOR_COMPACT, earliestPutTs);
>
>        }
>
>        if (scanner == null) {
>
>          Scan scan = new Scan();
>
>          scan.setMaxVersions(getFamily().getMaxVersions());
>
>          /* Include deletes, unless we are doing a major compaction */
>
>          scanner = new StoreScanner(this, getScanInfo(), scan, scanners,
>
>            majorCompaction? ScanType.MAJOR_COMPACT :
>
> ScanType.MINOR_COMPACT,
>
>            smallestReadPoint, earliestPutTs);
>
>        }
>
>        if (getHRegion().getCoprocessorHost() != null) {
>
>          InternalScanner cpScanner >
>            getHRegion().getCoprocessorHost().preCompact(this, scanner);
>
>          // NULL scanner returned from coprocessor hooks means skip
>
> normal processing
>
>          if (cpScanner == null) {
>
>            return null;
>
>          }
>
>          scanner = cpScanner;
>
>        }
>
>
>
> Can I file a JIRA to add the Scan Type to the preCompact method?
>
>
> Thanks,
>
>
> Asaf
>
+
lars hofhansl 2013-01-29, 18:55
+
Ted Yu 2013-01-29, 19:29
+
lars hofhansl 2013-01-29, 19:58
+
Ted Yu 2013-01-29, 20:33
+
Ted Yu 2013-01-30, 02:53
+
ramkrishna vasudevan 2013-01-30, 03:54