Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> determining ZK latency issues


Copy link to this message
-
Re: determining ZK latency issues
Since you mentioned that you have a high load such that you need to
disable fsync

I would suggested that you might want to start monitoring JVM stats (via
JMX) such as heap or gc activities. You can enable detailed gc logging to
see if latency spike corresponds to gc activity or not.
--
Thawan Kooburat

On 8/8/13 11:40 AM, "Neha Narkhede" <[EMAIL PROTECTED]> wrote:

>Since this zookeeper cluster is a pre-prod cluster and is used heavily, we
>have turned the forceSync to OFF. So we don't see the fsync warnings on
>this zookeeper cluster.
>
>Thanks,
>Neha
>
>
>On Thu, Aug 8, 2013 at 11:18 AM, Martin Kou <[EMAIL PROTECTED]> wrote:
>
>> Do you see fsync() warnings in your ZooKeeper log? It could be disk
>>writes
>> taking too long.
>>
>> Best Regards,
>> Martin Kou
>>
>>
>> On Thu, Aug 8, 2013 at 10:37 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > We have a 3-node ZK cluster running 3.3.4 that are experiencing high
>> > latency. From "stat", we saw avg latency to be 300-400ms consistently.
>> > Outstanding requests jumps to ~600 from time to time. However, the
>> server's
>> > I/O and CPU are both low. If I start a new ZK client and do a few
>>writes,
>> > the observed response time in the client is also low (1ms).
>> >
>> > What's the best way to figure out why the reported latency in ZK
>>server
>> is
>> > high? Also, are  min/avg/max latency computed on a rolling time
>>window?
>> >
>> > Thanks,
>> >
>> > Jun
>> >
>>