Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> How to know it's time for a major compaction?


+
Otis Gospodnetic 2012-12-12, 20:15
+
Kevin Odell 2012-12-12, 20:19
+
Ted Yu 2012-12-12, 20:22
+
Kevin Odell 2012-12-12, 20:30
+
Ted Yu 2012-12-12, 20:36
+
Kevin Odell 2012-12-12, 20:50
Copy link to this message
-
Re: How to know it's time for a major compaction?
In RegionObserver, we already have the following hook:

  /**
   * Called after the region is reported as open to the master.
   * @param c the environment provided by the region server
   */
  void postOpen(final ObserverContext<RegionCoprocessorEnvironment> c);

Auto-compaction logic can be triggered through the above hook.

Cheers

On Wed, Dec 12, 2012 at 12:36 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Thanks confirmation, Kevin.
>
> I logged HBASE-7340 so that user doesn't have to script for compaction.
>
>
> On Wed, Dec 12, 2012 at 12:30 PM, Kevin O'dell <[EMAIL PROTECTED]>wrote:
>
>> Ted,
>>
>>   Yes, it is all done through scripting.
>>
>> On Wed, Dec 12, 2012 at 3:22 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > bq. they will major compact any region the balancer has moved.
>> >
>> > Interesting. Do they achieve this through scripting ?
>> >
>> > I think HBase can perform some follow-on action after region movement.
>> >
>> > Cheers
>> >
>> > On Wed, Dec 12, 2012 at 12:19 PM, Kevin O'dell <
>> [EMAIL PROTECTED]
>> > >wrote:
>> >
>> > > Otis,
>> > >
>> > > I think it depends on your use case.  Some use cases are extremely
>> > > dependent on locality and latency, so they will major compact any
>> region
>> > > the balancer has moved.  Some people follow the HFile count and will
>> > > trigger a major when there are x amount of files.  Others will just
>> > follow
>> > > best practices and do it daily.  Some do very little writes and only
>> > > compact once a week.
>> > >
>> > > On Wed, Dec 12, 2012 at 3:15 PM, Otis Gospodnetic <
>> > > [EMAIL PROTECTED]> wrote:
>> > >
>> > > > Hello,
>> > > >
>> > > > What are the main metrics/signals people use/look at to say "OK, I
>> > really
>> > > > need to do a major compaction now", other than "hm, performance is
>> not
>> > > > good, let's try major compaction tonight"?
>> > > >
>> > > > Thanks,
>> > > > Otis
>> > > > --
>> > > > HBASE Performance Monitoring - http://sematext.com/spm/index.html
>> > > > Search Analytics - http://sematext.com/search-analytics/index.html
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Kevin O'Dell
>> > > Customer Operations Engineer, Cloudera
>> > >
>> >
>>
>>
>>
>> --
>> Kevin O'Dell
>> Customer Operations Engineer, Cloudera
>>
>
>