Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?


+
Stack 2013-06-20, 18:09
Copy link to this message
-
Re: [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?
Ted Yu 2013-06-20, 20:41
I looked at the diff for the following four commits:

r1414308 | stack | 2012-11-28 02:33:28 +0800 (Wed, 28 Nov 2012) | 1 line

HBASE-7110 refactor the compaction selection and config code similarly to
0.89-fb changes; REAPPLY v9
------------------------------------------------------------------------
r1414000 | stack | 2012-11-27 13:23:14 +0800 (Tue, 27 Nov 2012) | 1 line

HBASE-7110 refactor the compaction selection and config code similarly to
0.89-fb changes; REVERT of original patch and ADDENDUM because applied old
patch originally, v8
------------------------------------------------------------------------
r1413995 | stack | 2012-11-27 12:48:38 +0800 (Tue, 27 Nov 2012) | 1 line

HBASE-7110 refactor the compaction selection and config code similarly to
0.89-fb changes; ADDENDUM to fix broke TestHeapSize
------------------------------------------------------------------------
r1413912 | stack | 2012-11-27 06:51:37 +0800 (Tue, 27 Nov 2012) | 1 line

HBASE-7110 refactor the compaction selection and config code similarly to
0.89-fb changes
------------------------------------------------------------------------
r1407725 | larsh | 2012-11-10 12:28:07 +0800 (Sat, 10 Nov 2012) | 1 line

HBASE-4583 Integrate RWCC with Append and Increment operations

Here is what I found:

$ svn diff -r 1407725:1414308
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
| grep "hbase.hstore.compaction.min"
-      conf.getInt("hbase.hstore.compaction.min",
-    LOG.info("hbase.hstore.compaction.min = " + this.minFilesToCompact);
-    this.minCompactSize = conf.getLong("hbase.hstore.compaction.min.size",
-   *  "hbase.hstore.compaction.min.size"
-   *  "hbase.hstore.compaction.min"

Cheers

On Thu, Jun 20, 2013 at 11:09 AM, Stack <[EMAIL PROTECTED]> wrote:

> I was reading an old thriller, "HBASE-3149 Make flush decisions per column
> family", and I got to the good bit where our NicolasS argues that per-CF
> flush is likely not needed because small files is fine actually as long as
> these small files are hoovered up quckly.  He mentioned
> the hbase.hstore.compaction.min.size config which we'd set to be equal to
> flush size and he argued that our default should be much lower -- 1/16th
> smaller -- so we always get rid of the small files first.
>
> The config. was removed here:
>
> Author: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> Committer: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> Parent: 2c0261b4e6571d627fb017338aeaf10089b75dab (HBASE-7060 Region load
> balancing by table does not handle the case where a table's region count is
> lower than the number of the RS in the cluster (Ted Yu and Tianying))
> Child:  7380036d88ed6c6ddfad4f4fc2ef617ab419d610 (HBASE-7055 port
> HBASE-6371 tier-based compaction from 0.89-fb to trunk - revert for further
> discussion)
> Branches: many (31)
> Follows:
> Precedes:
>
>     HBASE-7055 port HBASE-6371 tier-based compaction from 0.89-fb to trunk
> (Sergey)
>
> I was wondering if w/ our new compaction algos if we are making use of
> NicolasS's advice (informed by experience) or not?
>
> Thanks,
> St.Ack
>
+
Andrew Purtell 2013-06-20, 21:21
+
Sergey Shelukhin 2013-06-20, 21:41
+
Stack 2013-06-20, 22:43
+
Stack 2013-06-24, 15:59
+
Sergey Shelukhin 2013-06-24, 17:19
+
lars hofhansl 2013-06-24, 16:32
+
Sergey Shelukhin 2013-06-24, 17:20
+
lars hofhansl 2013-06-20, 18:35