Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - GC pause issues


+
Varun Sharma 2013-01-24, 17:40
+
谢良 2013-01-25, 02:07
+
Varun Sharma 2013-01-25, 02:30
+
Varun Sharma 2013-01-25, 05:02
Copy link to this message
-
Re: 答复: GC pause issues
Jack Levin 2013-01-25, 07:46
Generally, the larger the flush to harder the GC will work. Flush more
often to avoid this. What is your total heap size set at?
On Jan 24, 2013 9:02 PM, "Varun Sharma" <[EMAIL PROTECTED]> wrote:

> I do have significant block cache churn and this issue is typical
> correlated with a huge increase in read latencies - could that be the
> reason for this - mslab should be taking care of the memstore related heap
> fragmentation ? Has anyone seen issues with block cache churn ?
>
> On Thu, Jan 24, 2013 at 6:30 PM, Varun Sharma <[EMAIL PROTECTED]> wrote:
>
> > I am curious how reducing the memstore size would help - I have 6 regions
> > and 3G total for memstore - so I would like max out on that by having a
> > bigger flush size per region. Are you asking me to have more regions and
> > smaller memstore flush size instead ? How is that likely to help
> >
> >
> > On Thu, Jan 24, 2013 at 6:07 PM, 谢良 <[EMAIL PROTECTED]> wrote:
> >
> >> Hi Varun,
> >>
> >> Please note if you try to increase new generation size, then the
> "ParNew"
> >> time will be up accordingly, and CMS YGC is also a STW.
> >> could you have a try to reduce memstore size to a smaller value, e.g.
> >> 128m or 256m ?
> >>
> >> Regards,
> >> Liang
> >> ________________________________________
> >> 发件人: Varun Sharma [[EMAIL PROTECTED]]
> >> 发送时间: 2013年1月25日 1:40
> >> 收件人: [EMAIL PROTECTED]
> >> 主题: GC pause issues
> >>
> >> Hi,
> >>
> >> I have a region server which has the following logs. As you can see from
> >> the log, ParNew is sufficiently big (450M) and there are heavy writes
> >> going
> >> in. I am seeing 200ms pauses which eventually build up and there is a
> >> promotion failure. There is a parnew collection every 2-3 seconds so it
> >> fills up real fast. My memstore size is bigger 512m for flushes and 4
> >> regions per server. (overall size is 3G for all memstores) - I have
> mslab
> >> enabled
> >>
> >> 2013-01-24T13:08:16.870+0000: 63533.964: [GC 63533.964: [ParNew:
> >> 471841K->52416K(471872K), 0.2251880 secs] 8733008K->8445039K(12727104K),
> >> 0.2254100 secs] [Times: user=0.50 sys=0.18, real=0.22 secs]
> >> 2013-01-24T13:08:19.546+0000: 63536.639: [GC 63536.639: [ParNew:
> >> 461593K->52416K(471872K), 0.2812690 secs] 8854216K->8557572K(12727104K),
> >> 0.2814870 secs] [Times: user=0.66 sys=0.09, real=0.29 secs]
> >> 013-01-24T13:08:21.824+0000: 63538.917: [GC 63538.918: [ParNew:
> >> 442836K->52416K(471872K), 0.2781490 secs] 8947992K->8705355K(12727104K),
> >> 0.2783810 secs] [Times: user=0.58 sys=0.14, real=0.28 secs]
> >> 2013-01-24T13:08:22.122+0000: 63539.216: [GC [1 CMS-initial-mark:
> >> 8652939K(12255232K)] 8752914K(12727104K), 0.0365000 secs] [Times:
> >> user=0.02
> >> sys=0.00, real=0.04 secs]
> >> 2013-01-24T13:08:22.159+0000: 63539.253: [CMS-concurrent-mark-start]
> >> 2013-01-24T13:08:24.953+0000: 63542.047: [GC 63542.047: [ParNew:
> >> 471872K->52251K(471872K), 0.1611970 secs] 9124811K->8831437K(12727104K),
> >> 0.1614180 secs] [Times: user=0.37 sys=0.19, real=0.16 secs]
> >> 2013-01-24T13:08:26.434+0000: 63543.527: [CMS-concurrent-mark:
> 4.105/4.268
> >> secs] [Times: user=5.36 sys=0.34, real=4.27 secs]
> >> 2013-01-24T13:08:26.434+0000: 63543.527: [CMS-concurrent-preclean-start]
> >> 2013-01-24T13:08:26.597+0000: 63543.691: [CMS-concurrent-preclean:
> >> 0.133/0.163 secs] [Times: user=0.16 sys=0.05, real=0.17 secs]
> >> 2013-01-24T13:08:26.597+0000: 63543.691:
> >> [CMS-concurrent-abortable-preclean-start]
> >> 2013-01-24T13:08:27.401+0000: 63544.495:
> >> [CMS-concurrent-abortable-preclean: 0.792/0.804 secs] [Times: user=1.46
> >> sys=0.16, real=0.80 secs]
> >> 2013-01-24T13:08:27.403+0000: 63544.496: [GC[YG occupancy: 274458 K
> >> (471872
> >> K)]63544.496: [Rescan (parallel) , 0.0540730 secs]63544.551: [weak refs
> >> processing, 0.0001700 secs] [1 CMS-remark: 8779186K(12255232K)]
> >> 9053645K(12727104K), 0.0544410 secs] [Times: user=0.20 sys=0.01,
> real=0.06
> >> secs]
> >> 2013-01-24T13:08:27.458+0000: 63544.551: [CMS-concurrent-sweep-start]
+
Varun Sharma 2013-01-25, 17:55
+
Jack Levin 2013-01-26, 06:54
+
varun kumar 2013-01-24, 19:22