Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?


Copy link to this message
-
Re: [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?
The history of this change on trunk appears to be the commit of a removal
as pointed out by Stack, then a revert of that commit. Later the patch or a
similar one is applied - and reverted and applied again - as pointed out by
Ted. A bit confusing and incidental to the discussion of its possible
usefulness. Can we get back to that?
On Thu, Jun 20, 2013 at 1:41 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> I looked at the diff for the following four commits:
>
> r1414308 | stack | 2012-11-28 02:33:28 +0800 (Wed, 28 Nov 2012) | 1 line
>
> HBASE-7110 refactor the compaction selection and config code similarly to
> 0.89-fb changes; REAPPLY v9
> ------------------------------------------------------------------------
> r1414000 | stack | 2012-11-27 13:23:14 +0800 (Tue, 27 Nov 2012) | 1 line
>
> HBASE-7110 refactor the compaction selection and config code similarly to
> 0.89-fb changes; REVERT of original patch and ADDENDUM because applied old
> patch originally, v8
> ------------------------------------------------------------------------
> r1413995 | stack | 2012-11-27 12:48:38 +0800 (Tue, 27 Nov 2012) | 1 line
>
> HBASE-7110 refactor the compaction selection and config code similarly to
> 0.89-fb changes; ADDENDUM to fix broke TestHeapSize
> ------------------------------------------------------------------------
> r1413912 | stack | 2012-11-27 06:51:37 +0800 (Tue, 27 Nov 2012) | 1 line
>
> HBASE-7110 refactor the compaction selection and config code similarly to
> 0.89-fb changes
> ------------------------------------------------------------------------
> r1407725 | larsh | 2012-11-10 12:28:07 +0800 (Sat, 10 Nov 2012) | 1 line
>
> HBASE-4583 Integrate RWCC with Append and Increment operations
>
> Here is what I found:
>
> $ svn diff -r 1407725:1414308
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> | grep "hbase.hstore.compaction.min"
> -      conf.getInt("hbase.hstore.compaction.min",
> -    LOG.info("hbase.hstore.compaction.min = " + this.minFilesToCompact);
> -    this.minCompactSize = conf.getLong("hbase.hstore.compaction.min.size",
> -   *  "hbase.hstore.compaction.min.size"
> -   *  "hbase.hstore.compaction.min"
>
> Cheers
>
> On Thu, Jun 20, 2013 at 11:09 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > I was reading an old thriller, "HBASE-3149 Make flush decisions per
> column
> > family", and I got to the good bit where our NicolasS argues that per-CF
> > flush is likely not needed because small files is fine actually as long
> as
> > these small files are hoovered up quckly.  He mentioned
> > the hbase.hstore.compaction.min.size config which we'd set to be equal to
> > flush size and he argued that our default should be much lower -- 1/16th
> > smaller -- so we always get rid of the small files first.
> >
> > The config. was removed here:
> >
> > Author: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> > Committer: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> > Parent: 2c0261b4e6571d627fb017338aeaf10089b75dab (HBASE-7060 Region load
> > balancing by table does not handle the case where a table's region count
> is
> > lower than the number of the RS in the cluster (Ted Yu and Tianying))
> > Child:  7380036d88ed6c6ddfad4f4fc2ef617ab419d610 (HBASE-7055 port
> > HBASE-6371 tier-based compaction from 0.89-fb to trunk - revert for
> further
> > discussion)
> > Branches: many (31)
> > Follows:
> > Precedes:
> >
> >     HBASE-7055 port HBASE-6371 tier-based compaction from 0.89-fb to
> trunk
> > (Sergey)
> >
> > I was wondering if w/ our new compaction algos if we are making use of
> > NicolasS's advice (informed by experience) or not?
> >
> > Thanks,
> > St.Ack
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)