Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - GC and High CPU

Copy link to this message
GC and High CPU
Jack Levin 2011-05-15, 02:12
Greetings all -- We had some issues with huge GC counts that drive CPU
to 100% and rendering RS basically dead, here is some data:

 We are normal here, about 1 ParNew per second.

    67 2011-05-14T15:35
     75 2011-05-14T15:36
     70 2011-05-14T15:37
     74 2011-05-14T15:38
     73 2011-05-14T15:39
     73 2011-05-14T15:40
     69 2011-05-14T15:41
     68 2011-05-14T15:42
     76 2011-05-14T15:43
     70 2011-05-14T15:44
     68 2011-05-14T15:45
    322 2011-05-14T15:46
    562 2011-05-14T15:47  <------------ This is where its about 10 per second.
    567 2011-05-14T15:48
    563 2011-05-14T15:49
    562 2011-05-14T15:50
    561 2011-05-14T15:51
    513 2011-05-14T15:52

here is the jstack from the time of this:

http://pastebin.com/RYdwH8Gp  Unusually high BLOCKED threads.

This is config for hbase-env:

export HBASE_OPTS="$HBASE_OPTS -verbose:gc  -XX:+PrintGCDetails
-XX:+PrintGCDateStamps -XX:+HeapDumpOnOutOfMemoryError

Does any have any suggestions on how to manage this?