Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?


Copy link to this message
-
Re: [COMPACTIONS] Anyone seen hbase.hstore.compaction.min.size in trunk/0.95?
The history of this change on trunk appears to be the commit of a removal
as pointed out by Stack, then a revert of that commit. Later the patch or a
similar one is applied - and reverted and applied again - as pointed out by
Ted. A bit confusing and incidental to the discussion of its possible
usefulness. Can we get back to that?
On Thu, Jun 20, 2013 at 1:41 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> I looked at the diff for the following four commits:
>
> r1414308 | stack | 2012-11-28 02:33:28 +0800 (Wed, 28 Nov 2012) | 1 line
>
> HBASE-7110 refactor the compaction selection and config code similarly to
> 0.89-fb changes; REAPPLY v9
> ------------------------------------------------------------------------
> r1414000 | stack | 2012-11-27 13:23:14 +0800 (Tue, 27 Nov 2012) | 1 line
>
> HBASE-7110 refactor the compaction selection and config code similarly to
> 0.89-fb changes; REVERT of original patch and ADDENDUM because applied old
> patch originally, v8
> ------------------------------------------------------------------------
> r1413995 | stack | 2012-11-27 12:48:38 +0800 (Tue, 27 Nov 2012) | 1 line
>
> HBASE-7110 refactor the compaction selection and config code similarly to
> 0.89-fb changes; ADDENDUM to fix broke TestHeapSize
> ------------------------------------------------------------------------
> r1413912 | stack | 2012-11-27 06:51:37 +0800 (Tue, 27 Nov 2012) | 1 line
>
> HBASE-7110 refactor the compaction selection and config code similarly to
> 0.89-fb changes
> ------------------------------------------------------------------------
> r1407725 | larsh | 2012-11-10 12:28:07 +0800 (Sat, 10 Nov 2012) | 1 line
>
> HBASE-4583 Integrate RWCC with Append and Increment operations
>
> Here is what I found:
>
> $ svn diff -r 1407725:1414308
> hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
> | grep "hbase.hstore.compaction.min"
> -      conf.getInt("hbase.hstore.compaction.min",
> -    LOG.info("hbase.hstore.compaction.min = " + this.minFilesToCompact);
> -    this.minCompactSize = conf.getLong("hbase.hstore.compaction.min.size",
> -   *  "hbase.hstore.compaction.min.size"
> -   *  "hbase.hstore.compaction.min"
>
> Cheers
>
> On Thu, Jun 20, 2013 at 11:09 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > I was reading an old thriller, "HBASE-3149 Make flush decisions per
> column
> > family", and I got to the good bit where our NicolasS argues that per-CF
> > flush is likely not needed because small files is fine actually as long
> as
> > these small files are hoovered up quckly.  He mentioned
> > the hbase.hstore.compaction.min.size config which we'd set to be equal to
> > flush size and he argued that our default should be much lower -- 1/16th
> > smaller -- so we always get rid of the small files first.
> >
> > The config. was removed here:
> >
> > Author: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> > Committer: Zhihong Yu <[EMAIL PROTECTED]>  2012-10-30 13:14:01
> > Parent: 2c0261b4e6571d627fb017338aeaf10089b75dab (HBASE-7060 Region load
> > balancing by table does not handle the case where a table's region count
> is
> > lower than the number of the RS in the cluster (Ted Yu and Tianying))
> > Child:  7380036d88ed6c6ddfad4f4fc2ef617ab419d610 (HBASE-7055 port
> > HBASE-6371 tier-based compaction from 0.89-fb to trunk - revert for
> further
> > discussion)
> > Branches: many (31)
> > Follows:
> > Precedes:
> >
> >     HBASE-7055 port HBASE-6371 tier-based compaction from 0.89-fb to
> trunk
> > (Sergey)
> >
> > I was wondering if w/ our new compaction algos if we are making use of
> > NicolasS's advice (informed by experience) or not?
> >
> > Thanks,
> > St.Ack
> >
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB