Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> GC frequency


Copy link to this message
-
Re: 答复: 答复: GC frequency
This is a very interesting article indeed, it seems to say that for heap
size > 4-8GB - time of new gen collection could be dominated by size of the
heap rather than size of new generation. This is interesting and I have not
found any such guideline in hbase book etc.

Do you think the overall heap size needs to go down in this case ?

On Thu, Feb 21, 2013 at 12:50 AM, 谢良 <[EMAIL PROTECTED]> wrote:

> Here is a good formula to estimate:
> http://blog.ragozin.info/2011/06/understanding-gc-pauses-in-jvm-hotspots.html
>
> Hope it helpful:)
> ________________________________________
> 发件人: Varun Sharma [[EMAIL PROTECTED]]
> 发送时间: 2013年2月21日 16:22
> 收件人: [EMAIL PROTECTED]
> 主题: Re: 答复: GC frequency
>
> What do you mean by normal size heap ? Here is JVM settings
>
> -Xms11480m -Xmx11480m -XX:NewSize=512m -XX:MaxNewSize=512m
> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=60
> -XX:+UseCMSInitiatingOccupancyOnly -XX:ParallelGCThreads=4
>
> I was told that for a 4 core machine, it typically takes 200ms to clean out
> 512m - now, the only thing that I am afraid with reducing the size of new
> gen is higher frequency and the chances of more frequent promotion
> failures.
>
> On Thu, Feb 21, 2013 at 12:10 AM, 谢良 <[EMAIL PROTECTED]> wrote:
>
> > Of course, you'll hit the nightmarish "CMS fragement" easier if NewSize
> > too low:)
> > Generally speaking, most of YGC should be less than 5ms for a normal size
> > heap.
> > maybe your load is too high or there're vm options be misconfigured ?
> > ________________________________________
> > 发件人: Varun Sharma [[EMAIL PROTECTED]]
> > 发送时间: 2013年2月21日 15:32
> > 收件人: [EMAIL PROTECTED]
> > 主题: GC frequency
> >
> > Hi,
> >
> > I have a system tuned with new Gen 512M with a lot write load. The system
> > has 4 cores - ParNewGC and GCThreads is set to 4. I am using ConcMarkGC
> and
> > CMSInitiating fraction is set to 60 %. I am observing the 90th/99th
> > percentile of latency and see it highly correlated with GC pauses. There
> > are times when I have a GC pause of ~ 200 ms every 4 seconds - the tail
> > latency shoots up to 200 milliseconds for reads - most reads are being
> > served out of cache. Looking at the GC log and tail latency pattern,
> there
> > is direct correlation b/w the two. When the write load is low, and the GC
> > pauses are like 100-150 ms every 6 seconds, the tail latency improves.
> >
> > After seeing this behaviour, I am intent on reducing the NewSize to 256M
> > but I risk 100 ms pauses pretty much every 1-2 seconds and perhaps higher
> > chance of promotion failures (MSLAB etc. is on). Does anyone know if
> > frequent young gen collections can be a problem ?
> >
> > Thanks
> > Varun
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB