Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?


Copy link to this message
-
Re: [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?
Part of HBASE-7055 patch that we picked includes CompactionConfiguration
class, which uses a prefix for config values.
See ::getMinCompactSize on that class, it's still used in compaction.

On Thu, Jun 20, 2013 at 2:21 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> The history of this change on trunk appears to be the commit of a removal
> as pointed out by Stack, then a revert of that commit. Later the patch or a
> similar one is applied - and reverted and applied again - as pointed out by
> Ted. A bit confusing and incidental to the discussion of its possible
> usefulness. Can we get back to that?
>
>
>
>
> On Thu, Jun 20, 2013 at 1:41 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > I looked at the diff for the following four commits:
> >
> > r1414308 | stack | 2012-11-28 02:33:28 +0800 (Wed, 28 Nov 2012) | 1 line
> >
> > HBASE-7110 refactor the compaction selection and config code similarly to
> > 0.89-fb changes; REAPPLY v9
> > ------------------------------------------------------------------------
> > r1414000 | stack | 2012-11-27 13:23:14 +0800 (Tue, 27 Nov 2012) | 1 line
> >
> > HBASE-7110 refactor the compaction selection and config code similarly to
> > 0.89-fb changes; REVERT of original patch and ADDENDUM because applied
> old
> > patch originally, v8
> > ------------------------------------------------------------------------
> > r1413995 | stack | 2012-11-27 12:48:38 +0800 (Tue, 27 Nov 2012) | 1 line
> >
> > HBASE-7110 refactor the compaction selection and config code similarly to
> > 0.89-fb changes; ADDENDUM to fix broke TestHeapSize
> > ------------------------------------------------------------------------
> > r1413912 | stack | 2012-11-27 06:51:37 +0800 (Tue, 27 Nov 2012) | 1 line
> >
> > HBASE-7110 refactor the compaction selection and config code similarly to
> > 0.89-fb changes
> > ------------------------------------------------------------------------
> > r1407725 | larsh | 2012-11-10 12:28:07 +0800 (Sat, 10 Nov 2012) | 1 line
> >
> > HBASE-4583 Integrate RWCC with Append and Increment operations
> >
> > Here is what I found:
> >
> > $ svn diff -r 1407725:1414308
> >
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> > | grep "hbase.hstore.compaction.min"
> > -      conf.getInt("hbase.hstore.compaction.min",
> > -    LOG.info("hbase.hstore.compaction.min = " + this.minFilesToCompact);
> > -    this.minCompactSize > conf.getLong("hbase.hstore.compaction.min.size",
> > -   *  "hbase.hstore.compaction.min.size"
> > -   *  "hbase.hstore.compaction.min"
> >
> > Cheers
> >
> > On Thu, Jun 20, 2013 at 11:09 AM, Stack <[EMAIL PROTECTED]> wrote:
> >
> > > I was reading an old thriller, "HBASE-3149 Make flush decisions per
> > column
> > > family", and I got to the good bit where our NicolasS argues that
> per-CF
> > > flush is likely not needed because small files is fine actually as long
> > as
> > > these small files are hoovered up quckly.  He mentioned
> > > the hbase.hstore.compaction.min.size config which we'd set to be equal
> to
> > > flush size and he argued that our default should be much lower --
> 1/16th
> > > smaller -- so we always get rid of the small files first.
> > >
> > > The config. was removed here:
> > >
> > > Author: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> > > Committer: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> > > Parent: 2c0261b4e6571d627fb017338aeaf10089b75dab (HBASE-7060 Region
> load
> > > balancing by table does not handle the case where a table's region
> count
> > is
> > > lower than the number of the RS in the cluster (Ted Yu and Tianying))
> > > Child:  7380036d88ed6c6ddfad4f4fc2ef617ab419d610 (HBASE-7055 port
> > > HBASE-6371 tier-based compaction from 0.89-fb to trunk - revert for
> > further
> > > discussion)
> > > Branches: many (31)
> > > Follows:
> > > Precedes:
> > >
> > >     HBASE-7055 port HBASE-6371 tier-based compaction from 0.89-fb to
> > trunk
> > > (Sergey)
> > >
> > > I was wondering if w/ our new compaction algos if we are making use of