Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Getting the scan type at preCompact


+
Mesika, Asaf 2013-01-29, 13:23
+
ramkrishna vasudevan 2013-01-29, 14:45
Copy link to this message
-
Re: Getting the scan type at preCompact
But my scanner is a wrapper to the scanner passed on to me in preCompact.
This scanner is created in the Store object (StoreScanner) with arguments I
don't have at preCompactScannerOpen(). So I can't create it. I want the
vanilla scanner hbase uses is preCompact so I can wrap it with my behavior
but also I need to know whether its minor or major.
Currently I hacked my region observer so that in preCompactScannerOpen I
save the ScanType in my instance as a field variable (ThreadLocal to be on
the safe side) and in preCompact I read it and use it to build my wrapped
scanner.

Sent from my iPhone

On 29 בינו 2013, at 16:46, ramkrishna vasudevan <
[EMAIL PROTECTED]> wrote:

You can do the same with preCompactScannerOpen() hook right.
If the scanType is minor send a wrapped internalScanner say
MinorCompactionScanner.

If the scanType is major send a wrapped interanlScanner say
MajorCompactionScanner.

Does that solve your purpose? Because if preCompactScannerOpen returns a
scanner only that will  be used inside preCompact also.

Regards
Ram

On Tue, Jan 29, 2013 at 6:53 PM, Mesika, Asaf <[EMAIL PROTECTED]> wrote:

Hi,
In the RegionObserver.preCompactScannerOpen() method, one of the

parameters is scanType which enables me to know if the compaction is major

or minor.

In the preCompact() method I don't have that parameter.
In a region observer I wrote, I'm basically wrapping the InternalScanner

with my own Scanner.

My scanner should behave differently when its a a major or minor

compaction.
Due to this restriction I'm forced to use preCompactScannerOpen() and

create the StoreScanner, copy pasting the code that creates it in

Store.compactStore() (marked in green):
       if (getHRegion().getCoprocessorHost() != null) {

         scanner = getHRegion()

             .getCoprocessorHost()

             .preCompactScannerOpen(this, scanners,

                 majorCompaction ? ScanType.MAJOR_COMPACT :

ScanType.MINOR_COMPACT, earliestPutTs);

       }

       if (scanner == null) {

         Scan scan = new Scan();

         scan.setMaxVersions(getFamily().getMaxVersions());

         /* Include deletes, unless we are doing a major compaction */

         scanner = new StoreScanner(this, getScanInfo(), scan, scanners,

           majorCompaction? ScanType.MAJOR_COMPACT :

ScanType.MINOR_COMPACT,

           smallestReadPoint, earliestPutTs);

       }

       if (getHRegion().getCoprocessorHost() != null) {

         InternalScanner cpScanner
           getHRegion().getCoprocessorHost().preCompact(this, scanner);

         // NULL scanner returned from coprocessor hooks means skip

normal processing

         if (cpScanner == null) {

           return null;

         }

         scanner = cpScanner;

       }

Can I file a JIRA to add the Scan Type to the preCompact method?
Thanks,
Asaf
+
Ted Yu 2013-01-29, 18:33
+
lars hofhansl 2013-01-29, 18:55
+
Ted Yu 2013-01-29, 19:29
+
lars hofhansl 2013-01-29, 19:58
+
Ted Yu 2013-01-29, 20:33
+
Ted Yu 2013-01-30, 02:53
+
ramkrishna vasudevan 2013-01-30, 03:54
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB