Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> difference between major and minor compactions?


Copy link to this message
-
Re: difference between major and minor compactions?
Thanks, JM
It seems like the sole difference btwn major and minor compaction is the
number of files (to be all or just a subset of storefiles). It mentioned
very briefly in
http://hbase.apache.org/book<http://hbase.apache.org/book/regions.arch.html>that
"Sometimes a minor compaction will ... promote itself to being a major
compaction". What does "sometime" exactly mean here? or any policy in HBase
that allow application to specify when to promote a minor compaction to be
a major (like user or some monitoring service can specify now is offpeak
time?)
Yun

On Sat, Jun 22, 2013 at 8:51 AM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi Yun,
>
> Few links:
> - http://blog.cloudera.com/blog/2012/06/hbase-io-hfile-input-output/
> => There is a small paragraph about compactions which explain when
> they are triggered.
> - http://hbase.apache.org/book/regions.arch.html 9.7.6.5
>
> You are almost right. Only thing is that HBase doesn't know when is
> your offpeak, so a major compaction can be triggered anytime if the
> minor is promoted to be a major one.
>
> JM
>
> 2013/6/22 yun peng <[EMAIL PROTECTED]>:
> > Hi, All
> >
> > I am asking the different practices of major and minor compaction... My
> > current understanding is that minor compaction, triggered automatically,
> > usually run along with online query serving (but in background), so that
> it
> > is important to make it as lightweight as possible... to minimise
> downtime
> > (pause time) of online query.
> >
> > In contrast, the major compaction is invoked in  offpeak time and usually
> > can be assume to have resource exclusively. It may have a different
> > performance optimization goal...
> >
> > Correct me if wrong, but let me know if HBase does design different
> > compaction mechanism this way..?
> >
> > Regards,
> > Yun
>