Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?


Copy link to this message
-
Re: [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?
Part of HBASE-7055 patch that we picked includes CompactionConfiguration
class, which uses a prefix for config values.
See ::getMinCompactSize on that class, it's still used in compaction.

On Thu, Jun 20, 2013 at 2:21 PM, Andrew Purtell <[EMAIL PROTECTED]> wrote:

> The history of this change on trunk appears to be the commit of a removal
> as pointed out by Stack, then a revert of that commit. Later the patch or a
> similar one is applied - and reverted and applied again - as pointed out by
> Ted. A bit confusing and incidental to the discussion of its possible
> usefulness. Can we get back to that?
>
>
>
>
> On Thu, Jun 20, 2013 at 1:41 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > I looked at the diff for the following four commits:
> >
> > r1414308 | stack | 2012-11-28 02:33:28 +0800 (Wed, 28 Nov 2012) | 1 line
> >
> > HBASE-7110 refactor the compaction selection and config code similarly to
> > 0.89-fb changes; REAPPLY v9
> > ------------------------------------------------------------------------
> > r1414000 | stack | 2012-11-27 13:23:14 +0800 (Tue, 27 Nov 2012) | 1 line
> >
> > HBASE-7110 refactor the compaction selection and config code similarly to
> > 0.89-fb changes; REVERT of original patch and ADDENDUM because applied
> old
> > patch originally, v8
> > ------------------------------------------------------------------------
> > r1413995 | stack | 2012-11-27 12:48:38 +0800 (Tue, 27 Nov 2012) | 1 line
> >
> > HBASE-7110 refactor the compaction selection and config code similarly to
> > 0.89-fb changes; ADDENDUM to fix broke TestHeapSize
> > ------------------------------------------------------------------------
> > r1413912 | stack | 2012-11-27 06:51:37 +0800 (Tue, 27 Nov 2012) | 1 line
> >
> > HBASE-7110 refactor the compaction selection and config code similarly to
> > 0.89-fb changes
> > ------------------------------------------------------------------------
> > r1407725 | larsh | 2012-11-10 12:28:07 +0800 (Sat, 10 Nov 2012) | 1 line
> >
> > HBASE-4583 Integrate RWCC with Append and Increment operations
> >
> > Here is what I found:
> >
> > $ svn diff -r 1407725:1414308
> >
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> > | grep "hbase.hstore.compaction.min"
> > -      conf.getInt("hbase.hstore.compaction.min",
> > -    LOG.info("hbase.hstore.compaction.min = " + this.minFilesToCompact);
> > -    this.minCompactSize > conf.getLong("hbase.hstore.compaction.min.size",
> > -   *  "hbase.hstore.compaction.min.size"
> > -   *  "hbase.hstore.compaction.min"
> >
> > Cheers
> >
> > On Thu, Jun 20, 2013 at 11:09 AM, Stack <[EMAIL PROTECTED]> wrote:
> >
> > > I was reading an old thriller, "HBASE-3149 Make flush decisions per
> > column
> > > family", and I got to the good bit where our NicolasS argues that
> per-CF
> > > flush is likely not needed because small files is fine actually as long
> > as
> > > these small files are hoovered up quckly.  He mentioned
> > > the hbase.hstore.compaction.min.size config which we'd set to be equal
> to
> > > flush size and he argued that our default should be much lower --
> 1/16th
> > > smaller -- so we always get rid of the small files first.
> > >
> > > The config. was removed here:
> > >
> > > Author: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> > > Committer: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> > > Parent: 2c0261b4e6571d627fb017338aeaf10089b75dab (HBASE-7060 Region
> load
> > > balancing by table does not handle the case where a table's region
> count
> > is
> > > lower than the number of the RS in the cluster (Ted Yu and Tianying))
> > > Child:  7380036d88ed6c6ddfad4f4fc2ef617ab419d610 (HBASE-7055 port
> > > HBASE-6371 tier-based compaction from 0.89-fb to trunk - revert for
> > further
> > > discussion)
> > > Branches: many (31)
> > > Follows:
> > > Precedes:
> > >
> > >     HBASE-7055 port HBASE-6371 tier-based compaction from 0.89-fb to
> > trunk
> > > (Sergey)
> > >
> > > I was wondering if w/ our new compaction algos if we are making use of
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB