Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> manual trigger of major compaction does not seems to work on 0.92


Copy link to this message
-
Re: manual trigger of major compaction does not seems to work on 0.92
In 0.92, we can handle multiple compactions but we wait on major compact
if minor compactions are ongoing for the same store.  It should be
enqueued and work after the minor compaction finishes, it's just not
immediately enqueued.  See HBASE-5330 for some discussion on this.  There
should be a log message in the regionserver to verify the the
MAJOR_COMPACT request was received#16;.  The existing contract on major
compaction requests are that they are asynchronous & the request has
in-memory persistence.

On 2/24/12 4:32 AM, "Mikael Sitruk" <[EMAIL PROTECTED]> wrote:

>I triggered several time major_compaction between 22:30 and the time i
>wrote the original email 00:58
>From the log of one of the severs between those time no major compaction
>was running a major_compaction ended at 22:23 on the same server but this
>is because the number of file was high (after a load phase) and not
>manually triggered.
>On the system major_compaction are disabled.
>
>From the logs:
>1. 22:23:42 --> 22:23:48 - single major compaction runs
>2. 22:23:48 --> 23:44:27 - minor compaction runs
>3. 23:44:27 --> 23:44:32 - single major compaction runs
>5. 23:44:32 --> 01:13:49 - minor compaction are run
>5. 01:13:49 the cycle of major compaction began to run (heavily working
>only from 01:40:).
>-------------------------------
>*below a grep of compaction (ends, with major and minor to illustrate the
>above)*
>
>2012-02-23 22:23:48,272 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 1 file(s) in PI of
>TC,00000000000012921999,1329399349811.8dcc952ff57e87c6d82840e84eb870a6.
>into 7400a662b5924d3baf3ac520b1eb7ad9, size=49.0m; total size for store is
>49.0m
>2012-02-23 22:25:38,477 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000010451999,1329399349799.38739d7acb5f10198540bcfa0d4b5aef.
>into 77c43127e38846f78e7ada302d399732, size=540.8m; total size for store
>is
>2.1g
>2012-02-23 22:27:41,930 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000003951999,1329399349764.f55cfda6067ad1425ba68d20d82c945b.
>into f9c31ac78b3d4feb848b1e8d78975e34, size=532.0m; total size for store
>is
>2.1g
>2012-02-23 22:29:43,260 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000010971999,1329399349802.15720106584fe32f58287b6174970f31.
>into af2472f30a9f4c558323a3268f98183c, size=517.9m; total size for store
>is
>2.6g
>2012-02-23 22:30:19,476 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000010061999,1329399349797.27a2f37d5d6ac48c860cca5251be5731.
>into 3ac255c439ab4c518f4a938d542c9220, size=335.4m; total size for store
>is
>2.1g
>2012-02-23 22:31:15,051 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000008501999,1329399349789.fa0ac64d8ac9050f4acf2b0deb6e2dc7.
>into b8c733a27d02468a82f8a2c4d84db4ff, size=523.0m; total size for store
>is
>2.1g
>2012-02-23 22:32:33,430 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000004991999,1329399349770.46a2a16d484faacfcda36bc52bb218ba.
>into 5344911081424de69299128412d41573, size=538.0m; total size for store
>is
>2.2g
>2012-02-23 22:33:28,882 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000004081999,1329399349765.a5ca0a6a0a5561af5b6964e458615493.
>into d15c1e5b089d46beb2c4d01c796367a7, size=520.7m; total size for store
>is
>2.1g
>2012-02-23 22:34:46,714 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000007591999,1329399349784.711b032969e1ae6cce8e16bb9b
>...
>..
>..
>2012-02-23 23:44:32,215 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 1 file(s) in PI of
>TC,00000000000000051999,1329399349734.e7ba4a35ee8524fbf4e6f9f9ba845206.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB