Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> GC recommendations for large Region Server heaps


Copy link to this message
-
Re: GC recommendations for large Region Server heaps
Thanks Anil - your setting matches our current JVM setting quite closely
(though ours is for a 8GB heap).

--Suraj
On Tue, Jul 9, 2013 at 1:53 PM, anil gupta <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am using 34 GB heap and my use case is also Read oriented. I also do
> writes with MR. :)
> Yesterday, under decent load i got pauses of 30-40 secs. Still, the RS were
> not using the full 34 GB. I am thinking of doing some more tuning as i
> expect the read load to increase.
>
> Here is my GC setting for JDK6: -XX:NewSize=200m -XX:MaxNewSize=400m
> -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
> -XX:CMSInitiatingOccupancyFraction=70 -verbose:gc
>
> ~Anil
>
>
> On Tue, Jul 9, 2013 at 8:12 AM, Bryan Beaudreault
> <[EMAIL PROTECTED]>wrote:
>
> > I see from the blog post that it is java7.  The question still stands
> > regarding using that with hbase, considering the open jira
> > https://issues.apache.org/jira/browse/HBASE-5261
> >
> >
> > On Tue, Jul 9, 2013 at 11:03 AM, Bryan Beaudreault <
> > [EMAIL PROTECTED]
> > > wrote:
> >
> > > @Otis, are you guys running G1GC with java6 or java7? From what I'm
> > > reading it seems to be more stable with better performance in java7,
> but
> > I
> > > also believe java7 is not officially supported by apache hadoop or
> hbase
> > > yet.  I'm wondering if many people are using java7 for hbase without
> > issue
> > > despite the lack of support.
> > >
> > >
> > > On Tue, Jul 9, 2013 at 1:52 AM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
> > >
> > >> This is my HBASE GC options of CMS, it does work well.
> > >>
> > >> XX:+DisableExplicitGC -XX:+UseCompressedOops -XX:PermSize=160m
> > >> -XX:MaxPermSize=160m -XX:GCTimeRatio=19 -XX:SoftRefLRUPolicyMSPerMB=0
> > >> -XX:SurvivorRatio=2 -XX:MaxTenuringThreshold=1
> > -XX:+UseFastAccessorMethods
> > >> -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled
> > >> -XX:CMSInitiatingOccupancyFraction=70
> -XX:+UseCMSCompactAtFullCollection
> > >> -XX:CMSFullGCsBeforeCompaction=0 -XX:+CMSClassUnloadingEnabled
> > >> -XX:CMSMaxAbortablePrecleanTime=300 -XX:+CMSScavengeBeforeRemark
> > >>
> > >>
> > >>
> > >> On Tue, Jul 9, 2013 at 1:12 PM, Otis Gospodnetic <
> > >> [EMAIL PROTECTED]
> > >> > wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > Check
> > >> http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/
> > >> >
> > >> > Those graphs show RegionServer before and after switch to G1.  The
> > >> > dashboard screenshot further below shows CMS (top row) vs. G1
> (bottom
> > >> > row).  After those tests we ended up switching to G1 across the
> whole
> > >> > cluster and haven't had issues or major pauses since.... knock on
> > >> > keyboard.
> > >> >
> > >> > Otis
> > >> > --
> > >> > Solr & ElasticSearch Support -- http://sematext.com/
> > >> > Performance Monitoring -- http://sematext.com/spm
> > >> >
> > >> >
> > >> >
> > >> > On Mon, Jul 8, 2013 at 2:56 PM, Stack <[EMAIL PROTECTED]> wrote:
> > >> > > On Mon, Jul 8, 2013 at 11:09 AM, Suraj Varma <[EMAIL PROTECTED]
> >
> > >> > wrote:
> > >> > >
> > >> > >> Hello:
> > >> > >> We have an HBase cluster with region servers running on 8GB heap
> > size
> > >> > with
> > >> > >> a 0.6 block cache (it is a read heavy cluster, with bursty write
> > >> traffic
> > >> > >> via MR jobs). (version: hbase-0.94.6.1)
> > >> > >>
> > >> > >> During HBaseCon, while speaking to a few attendees, I heard some
> > >> folks
> > >> > were
> > >> > >> running region servers as high as 24GB and some others in the
> 16GB
> > >> > range.
> > >> > >>
> > >> > >> So - question: Are there any special GC recommendations (tuning
> > >> > parameters,
> > >> > >> flags, etc) that folks who run at these large heaps can recommend
> > >> while
> > >> > >> moving up from an 8GB heap? i.e. for 16GB and for 24GB RS heaps
> > ... ?
> > >> > >>
> > >> > >> I'm especially concerned about long pauses causing zk session
> > >> timeouts
> > >> > and
> > >> > >> consequent RS shutdowns. Our boxes do have a lot of RAM and we