Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> GC recommendations for large Region Server heaps


Copy link to this message
-
Re: GC recommendations for large Region Server heaps
Hey Suraj,

  I would recommend turning on MSlab and using the following settings:

-XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled

On Mon, Jul 8, 2013 at 2:09 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:

> Hello:
> We have an HBase cluster with region servers running on 8GB heap size with
> a 0.6 block cache (it is a read heavy cluster, with bursty write traffic
> via MR jobs). (version: hbase-0.94.6.1)
>
> During HBaseCon, while speaking to a few attendees, I heard some folks were
> running region servers as high as 24GB and some others in the 16GB range.
>
> So - question: Are there any special GC recommendations (tuning parameters,
> flags, etc) that folks who run at these large heaps can recommend while
> moving up from an 8GB heap? i.e. for 16GB and for 24GB RS heaps ... ?
>
> I'm especially concerned about long pauses causing zk session timeouts and
> consequent RS shutdowns. Our boxes do have a lot of RAM and we are
> exploring how we can use more of it for the cluster while maintaining
> overall stability.
>
> Also - if there are clusters running multiple region servers per host, I'd
> be very interested to know what RS heap sizes those are being run at ...
> and whether this was chosen as an alternative to running a single RS with
> large heap.
>
> (I know I'll have to test the GC stuff out on my cluster and for my
> workloads anyway ... but just trying to get a feel of what sort of tuning
> options had to be used to have a stable HBase cluster with 16 or 24GB RS
> heaps).
>
> Thanks in advance,
> --Suraj
>

--
Kevin O'Dell
Systems Engineer, Cloudera