Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Getting the scan type at preCompact


+
Mesika, Asaf 2013-01-29, 13:23
+
ramkrishna vasudevan 2013-01-29, 14:45
+
Asaf Mesika 2013-01-29, 18:21
+
Ted Yu 2013-01-29, 18:33
+
lars hofhansl 2013-01-29, 18:55
+
Ted Yu 2013-01-29, 19:29
+
lars hofhansl 2013-01-29, 19:58
+
Ted Yu 2013-01-29, 20:33
Copy link to this message
-
Re: Getting the scan type at preCompact
Ted Yu 2013-01-30, 02:53
I have checked in the following into trunk:
HBASE-7712 Pass ScanType into preCompact()

Since I didn't find a clean way of introducing ScanType into 0.94 without
requiring recompilation / redeployment of coprocessor jar, the above change
is only in 0.96

Cheers

On Tue, Jan 29, 2013 at 12:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> I have logged HBASE-7712
>
> Patch coming soon.
>
>
> On Tue, Jan 29, 2013 at 11:58 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
>> Oh yeah, wasn't disagreeing with that, just that until we have a patch
>> there's a workaround.
>>
>>   ------------------------------
>> *From:* Ted Yu <[EMAIL PROTECTED]>
>> *To:* [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
>> *Sent:* Tuesday, January 29, 2013 11:29 AM
>> *Subject:* Re: Getting the scan type at preCompact
>>
>> Since preCompact() is given the scanner returned
>> from preCompactScannerOpen(), I think for users it is more convenient to
>> wrap this scanner in preCompact().
>>
>> Cheers
>>
>> On Tue, Jan 29, 2013 at 10:55 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>>
>> I added the preCompactScannerOpen() to RegionObserver and didn't go back
>> and also changes preCompact.
>> In lieu of a fix you could create the scanner in preCompactScannerOpen()
>> (take a look at Compactor.compact() to see how the scanner would be
>> created).
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>>  From: "Mesika, Asaf" <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Sent: Tuesday, January 29, 2013 5:23 AM
>> Subject: Getting the scan type at preCompact
>>
>> Hi,
>>
>> In the RegionObserver.preCompactScannerOpen() method, one of the
>> parameters is scanType which enables me to know if the compaction is major
>> or minor.
>> In the preCompact() method I don't have that parameter.
>>
>> In a region observer I wrote, I'm basically wrapping the InternalScanner
>> with my own Scanner.
>> My scanner should behave differently when its a a major or minor
>> compaction.
>>
>> Due to this restriction I'm forced to use preCompactScannerOpen() and
>> create the StoreScanner, copy pasting the code that creates it in
>> Store.compactStore() (marked in green):
>>
>>         if (getHRegion().getCoprocessorHost() != null) {
>>           scanner = getHRegion()
>>               .getCoprocessorHost()
>>               .preCompactScannerOpen(this, scanners,
>>                   majorCompaction ? ScanType.MAJOR_COMPACT :
>> ScanType.MINOR_COMPACT, earliestPutTs);
>>         }
>>         if (scanner == null) {
>>           Scan scan = new Scan();
>>           scan.setMaxVersions(getFamily().getMaxVersions());
>>           /* Include deletes, unless we are doing a major compaction */
>>           scanner = new StoreScanner(this, getScanInfo(), scan, scanners,
>>             majorCompaction? ScanType.MAJOR_COMPACT :
>> ScanType.MINOR_COMPACT,
>>             smallestReadPoint, earliestPutTs);
>>         }
>>         if (getHRegion().getCoprocessorHost() != null) {
>>           InternalScanner cpScanner >>             getHRegion().getCoprocessorHost().preCompact(this, scanner);
>>           // NULL scanner returned from coprocessor hooks means skip
>> normal processing
>>           if (cpScanner == null) {
>>             return null;
>>           }
>>           scanner = cpScanner;
>>         }
>>
>>
>> Can I file a JIRA to add the Scan Type to the preCompact method?
>>
>> Thanks,
>>
>> Asaf
>>
>>
>>
>>
>>
>
+
ramkrishna vasudevan 2013-01-30, 03:54