Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Tables gets Major Compacted even if they haven't changed


+
Premal Shah 2013-09-10, 04:02
+
lars hofhansl 2013-09-10, 05:41
+
Dave Latham 2013-09-10, 18:11
+
Premal Shah 2013-09-10, 18:39
+
lars hofhansl 2013-09-11, 02:12
+
Vladimir Rodionov 2013-09-10, 04:13
+
Premal Shah 2013-09-10, 04:28
+
Vladimir Rodionov 2013-09-10, 05:10
Copy link to this message
-
Re: Tables gets Major Compacted even if they haven't changed
Hi Premal,

You can set hbase.hregion.majorcompaction=0 i.e. never run major compaction
by itself. Then major compaction will never run periodically by itself.
Either, user has to trigger it manually or this will be driven by no. of
store files.

HTH,
Anil
On Mon, Sep 9, 2013 at 9:28 PM, Premal Shah <[EMAIL PROTECTED]> wrote:

> Ah ok. We don't expire any data, so have not set any TTLs. Is there a
> policy we can use to avoid compacting regions that have not changed (ie
> have just 1 store file)?
>
>
> On Mon, Sep 9, 2013 at 9:13 PM, Vladimir Rodionov
> <[EMAIL PROTECTED]>wrote:
>
> > HBase can run major compaction (even if table has not been updated) to
> > purge expired data (TTL).
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: [EMAIL PROTECTED]
> >
> > ________________________________________
> > From: Premal Shah [[EMAIL PROTECTED]]
> > Sent: Monday, September 09, 2013 9:02 PM
> > To: user
> > Subject: Tables gets Major Compacted even if they haven't changed
> >
> > Hi,
> > We have a bunch on tables in our HBase cluster. We have a script which
> > makes sure all of them get Major Compacted once every 2 days. There are 2
> > things I'm observing
> >
> > 1) Table X has not updated in a month. We have not inserted, updated or
> > deleted data. However, it still major compacts every 2 days. All the
> > regions in this table have only 1 store file.
> >
> > 2) Table Y has a few regions where the rowkey is essentially a timestamp.
> > So, we only write to 1 region at a time. Over time, the region splits,
> and
> > then we write the one of the split regions. Now, whenever we major
> compact
> > the table, all regions get major compacted. Only 1 region has more than 1
> > store file, every other region has exactly once.
> >
> > Is there a way to avoid compaction of regions that have not changed?
> >
> > We are using HBase 0.94.11
> >
> > --
> > Regards,
> > Premal Shah.
> >
> > Confidentiality Notice:  The information contained in this message,
> > including any attachments hereto, may be confidential and is intended to
> be
> > read only by the individual or entity to whom this message is addressed.
> If
> > the reader of this message is not the intended recipient or an agent or
> > designee of the intended recipient, please note that any review, use,
> > disclosure or distribution of this message or its attachments, in any
> form,
> > is strictly prohibited.  If you have received this message in error,
> please
> > immediately notify the sender and/or [EMAIL PROTECTED] and
> > delete or destroy any copy of this message and its attachments.
> >
>
>
>
> --
> Regards,
> Premal Shah.
>

--
Thanks & Regards,
Anil Gupta
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB