Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?


Copy link to this message
-
Re: [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?
I looked at the diff for the following four commits:

r1414308 | stack | 2012-11-28 02:33:28 +0800 (Wed, 28 Nov 2012) | 1 line

HBASE-7110 refactor the compaction selection and config code similarly to
0.89-fb changes; REAPPLY v9
------------------------------------------------------------------------
r1414000 | stack | 2012-11-27 13:23:14 +0800 (Tue, 27 Nov 2012) | 1 line

HBASE-7110 refactor the compaction selection and config code similarly to
0.89-fb changes; REVERT of original patch and ADDENDUM because applied old
patch originally, v8
------------------------------------------------------------------------
r1413995 | stack | 2012-11-27 12:48:38 +0800 (Tue, 27 Nov 2012) | 1 line

HBASE-7110 refactor the compaction selection and config code similarly to
0.89-fb changes; ADDENDUM to fix broke TestHeapSize
------------------------------------------------------------------------
r1413912 | stack | 2012-11-27 06:51:37 +0800 (Tue, 27 Nov 2012) | 1 line

HBASE-7110 refactor the compaction selection and config code similarly to
0.89-fb changes
------------------------------------------------------------------------
r1407725 | larsh | 2012-11-10 12:28:07 +0800 (Sat, 10 Nov 2012) | 1 line

HBASE-4583 Integrate RWCC with Append and Increment operations

Here is what I found:

$ svn diff -r 1407725:1414308
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
| grep "hbase.hstore.compaction.min"
-      conf.getInt("hbase.hstore.compaction.min",
-    LOG.info("hbase.hstore.compaction.min = " + this.minFilesToCompact);
-    this.minCompactSize = conf.getLong("hbase.hstore.compaction.min.size",
-   *  "hbase.hstore.compaction.min.size"
-   *  "hbase.hstore.compaction.min"

Cheers

On Thu, Jun 20, 2013 at 11:09 AM, Stack <[EMAIL PROTECTED]> wrote:

> I was reading an old thriller, "HBASE-3149 Make flush decisions per column
> family", and I got to the good bit where our NicolasS argues that per-CF
> flush is likely not needed because small files is fine actually as long as
> these small files are hoovered up quckly.  He mentioned
> the hbase.hstore.compaction.min.size config which we'd set to be equal to
> flush size and he argued that our default should be much lower -- 1/16th
> smaller -- so we always get rid of the small files first.
>
> The config. was removed here:
>
> Author: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> Committer: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> Parent: 2c0261b4e6571d627fb017338aeaf10089b75dab (HBASE-7060 Region load
> balancing by table does not handle the case where a table's region count is
> lower than the number of the RS in the cluster (Ted Yu and Tianying))
> Child:  7380036d88ed6c6ddfad4f4fc2ef617ab419d610 (HBASE-7055 port
> HBASE-6371 tier-based compaction from 0.89-fb to trunk - revert for further
> discussion)
> Branches: many (31)
> Follows:
> Precedes:
>
>     HBASE-7055 port HBASE-6371 tier-based compaction from 0.89-fb to trunk
> (Sergey)
>
> I was wondering if w/ our new compaction algos if we are making use of
> NicolasS's advice (informed by experience) or not?
>
> Thanks,
> St.Ack
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB