Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Major compaction issues in hbase


Copy link to this message
-
Re: Major compaction issues in hbase
Vladimir Rodionov 2014-02-02, 21:56
I think, the major issue here - not a sudden major compaction, but why
everything, literally, stalls when major compaction kicks off? Is there any
clear explanation, somewhere, how does major compaction can affect tput,
latency, availability of HBase? 300ms latency is possible, of course , as
an rare outlier, but if its 90% percentile - its totally different story.

-Vladimir
On Sun, Feb 2, 2014 at 1:49 PM, Kevin O'dell <[EMAIL PROTECTED]>wrote:

> Hi Y,
>
>   That may help, but a coulple quick questions:
>
> Are you doing any bulk loads?
>
> What is your avg flush size through out the day?
>
> After a day before how many storefiles have you built up?
>
> I would look at your memory sizing along with scheduling daily compactions.
>
>
> On Sun, Feb 2, 2014 at 4:14 PM, yanivG <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> > I am using hbase 0.94.6 version. We are developing a java application and
> > using it as our storage for a high load web application.
> > The average ration is about 500 requests per second.
> > From time to time I see high latency (more then 300 ms!!!) when trying to
> > store data object within hbase (using Put).
> >
> > We disabled major compaction and set it to run manually once a week.
> > By examining the logs during those high latency , I see that major
> > compaction was executed.
> > I saw a post
> > (https://groups.google.com/forum/#!topic/nosql-databases/YwE894gq1qM)
> > which
> > explains it.
> > The question is how can I avoid this? Should I ran the major compaction
> > once
> > a day in low load hours? will it eliminate the sporadic major compaction?
> > Thanks,
> > Y
> >
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-hbase.679495.n3.nabble.com/Major-compaction-issues-in-hbase-tp4055592.html
> > Sent from the HBase Developer mailing list archive at Nabble.com.
> >
>
>
>
> --
> Kevin O'Dell
> Systems Engineer, Cloudera
>