Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - GC recommendations for large Region Server heaps


+
Suraj Varma 2013-07-08, 18:09
+
Vladimir Rodionov 2013-07-08, 18:33
+
Suraj Varma 2013-07-08, 18:44
Copy link to this message
-
Re: GC recommendations for large Region Server heaps
Kevin O'dell 2013-07-08, 18:25
Hey Suraj,

  I would recommend turning on MSlab and using the following settings:

-XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled

On Mon, Jul 8, 2013 at 2:09 PM, Suraj Varma <[EMAIL PROTECTED]> wrote:

> Hello:
> We have an HBase cluster with region servers running on 8GB heap size with
> a 0.6 block cache (it is a read heavy cluster, with bursty write traffic
> via MR jobs). (version: hbase-0.94.6.1)
>
> During HBaseCon, while speaking to a few attendees, I heard some folks were
> running region servers as high as 24GB and some others in the 16GB range.
>
> So - question: Are there any special GC recommendations (tuning parameters,
> flags, etc) that folks who run at these large heaps can recommend while
> moving up from an 8GB heap? i.e. for 16GB and for 24GB RS heaps ... ?
>
> I'm especially concerned about long pauses causing zk session timeouts and
> consequent RS shutdowns. Our boxes do have a lot of RAM and we are
> exploring how we can use more of it for the cluster while maintaining
> overall stability.
>
> Also - if there are clusters running multiple region servers per host, I'd
> be very interested to know what RS heap sizes those are being run at ...
> and whether this was chosen as an alternative to running a single RS with
> large heap.
>
> (I know I'll have to test the GC stuff out on my cluster and for my
> workloads anyway ... but just trying to get a feel of what sort of tuning
> options had to be used to have a stable HBase cluster with 16 or 24GB RS
> heaps).
>
> Thanks in advance,
> --Suraj
>

--
Kevin O'Dell
Systems Engineer, Cloudera
+
Stack 2013-07-08, 18:56
+
Otis Gospodnetic 2013-07-09, 05:12
+
Azuryy Yu 2013-07-09, 05:52
+
Bryan Beaudreault 2013-07-09, 15:03
+
Otis Gospodnetic 2013-07-09, 23:15
+
Bryan Beaudreault 2013-07-09, 15:12
+
anil gupta 2013-07-09, 20:53
+
Suraj Varma 2013-07-10, 00:07
+
Suraj Varma 2013-07-10, 00:05
+
Azuryy Yu 2013-07-10, 01:08
+
Varun Sharma 2013-07-10, 03:20
+
Azuryy Yu 2013-07-10, 03:24
+
Suraj Varma 2013-07-10, 00:01
+
Suraj Varma 2013-07-09, 23:57