Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Getting the scan type at preCompact


+
Mesika, Asaf 2013-01-29, 13:23
+
ramkrishna vasudevan 2013-01-29, 14:45
+
Asaf Mesika 2013-01-29, 18:21
+
Ted Yu 2013-01-29, 18:33
+
lars hofhansl 2013-01-29, 18:55
+
Ted Yu 2013-01-29, 19:29
+
lars hofhansl 2013-01-29, 19:58
Copy link to this message
-
Re: Getting the scan type at preCompact
I have logged HBASE-7712

Patch coming soon.

On Tue, Jan 29, 2013 at 11:58 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:

> Oh yeah, wasn't disagreeing with that, just that until we have a patch
> there's a workaround.
>
>   ------------------------------
> *From:* Ted Yu <[EMAIL PROTECTED]>
> *To:* [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
> *Sent:* Tuesday, January 29, 2013 11:29 AM
> *Subject:* Re: Getting the scan type at preCompact
>
> Since preCompact() is given the scanner returned
> from preCompactScannerOpen(), I think for users it is more convenient to
> wrap this scanner in preCompact().
>
> Cheers
>
> On Tue, Jan 29, 2013 at 10:55 AM, lars hofhansl <[EMAIL PROTECTED]> wrote:
>
> I added the preCompactScannerOpen() to RegionObserver and didn't go back
> and also changes preCompact.
> In lieu of a fix you could create the scanner in preCompactScannerOpen()
> (take a look at Compactor.compact() to see how the scanner would be
> created).
>
> -- Lars
>
>
>
> ________________________________
>  From: "Mesika, Asaf" <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Tuesday, January 29, 2013 5:23 AM
> Subject: Getting the scan type at preCompact
>
> Hi,
>
> In the RegionObserver.preCompactScannerOpen() method, one of the
> parameters is scanType which enables me to know if the compaction is major
> or minor.
> In the preCompact() method I don't have that parameter.
>
> In a region observer I wrote, I'm basically wrapping the InternalScanner
> with my own Scanner.
> My scanner should behave differently when its a a major or minor
> compaction.
>
> Due to this restriction I'm forced to use preCompactScannerOpen() and
> create the StoreScanner, copy pasting the code that creates it in
> Store.compactStore() (marked in green):
>
>         if (getHRegion().getCoprocessorHost() != null) {
>           scanner = getHRegion()
>               .getCoprocessorHost()
>               .preCompactScannerOpen(this, scanners,
>                   majorCompaction ? ScanType.MAJOR_COMPACT :
> ScanType.MINOR_COMPACT, earliestPutTs);
>         }
>         if (scanner == null) {
>           Scan scan = new Scan();
>           scan.setMaxVersions(getFamily().getMaxVersions());
>           /* Include deletes, unless we are doing a major compaction */
>           scanner = new StoreScanner(this, getScanInfo(), scan, scanners,
>             majorCompaction? ScanType.MAJOR_COMPACT :
> ScanType.MINOR_COMPACT,
>             smallestReadPoint, earliestPutTs);
>         }
>         if (getHRegion().getCoprocessorHost() != null) {
>           InternalScanner cpScanner >             getHRegion().getCoprocessorHost().preCompact(this, scanner);
>           // NULL scanner returned from coprocessor hooks means skip
> normal processing
>           if (cpScanner == null) {
>             return null;
>           }
>           scanner = cpScanner;
>         }
>
>
> Can I file a JIRA to add the Scan Type to the preCompact method?
>
> Thanks,
>
> Asaf
>
>
>
>
>
+
Ted Yu 2013-01-30, 02:53
+
ramkrishna vasudevan 2013-01-30, 03:54
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB