Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - How to know it's time for a major compaction?


+
Otis Gospodnetic 2012-12-12, 20:15
+
Kevin Odell 2012-12-12, 20:19
+
Ted Yu 2012-12-12, 20:22
+
Kevin Odell 2012-12-12, 20:30
+
Ted Yu 2012-12-12, 20:36
Copy link to this message
-
Re: How to know it's time for a major compaction?
Kevin O'dell 2012-12-12, 20:50
Ted,

  That is awesome.  Thank you!

On Wed, Dec 12, 2012 at 3:36 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Thanks confirmation, Kevin.
>
> I logged HBASE-7340 so that user doesn't have to script for compaction.
>
> On Wed, Dec 12, 2012 at 12:30 PM, Kevin O'dell <[EMAIL PROTECTED]
> >wrote:
>
> > Ted,
> >
> >   Yes, it is all done through scripting.
> >
> > On Wed, Dec 12, 2012 at 3:22 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> > > bq. they will major compact any region the balancer has moved.
> > >
> > > Interesting. Do they achieve this through scripting ?
> > >
> > > I think HBase can perform some follow-on action after region movement.
> > >
> > > Cheers
> > >
> > > On Wed, Dec 12, 2012 at 12:19 PM, Kevin O'dell <
> [EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Otis,
> > > >
> > > > I think it depends on your use case.  Some use cases are extremely
> > > > dependent on locality and latency, so they will major compact any
> > region
> > > > the balancer has moved.  Some people follow the HFile count and will
> > > > trigger a major when there are x amount of files.  Others will just
> > > follow
> > > > best practices and do it daily.  Some do very little writes and only
> > > > compact once a week.
> > > >
> > > > On Wed, Dec 12, 2012 at 3:15 PM, Otis Gospodnetic <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > What are the main metrics/signals people use/look at to say "OK, I
> > > really
> > > > > need to do a major compaction now", other than "hm, performance is
> > not
> > > > > good, let's try major compaction tonight"?
> > > > >
> > > > > Thanks,
> > > > > Otis
> > > > > --
> > > > > HBASE Performance Monitoring - http://sematext.com/spm/index.html
> > > > > Search Analytics - http://sematext.com/search-analytics/index.html
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Kevin O'Dell
> > > > Customer Operations Engineer, Cloudera
> > > >
> > >
> >
> >
> >
> > --
> > Kevin O'Dell
> > Customer Operations Engineer, Cloudera
> >
>

--
Kevin O'Dell
Customer Operations Engineer, Cloudera
+
Ted Yu 2012-12-15, 05:53