-
Re: ZK fsync warningPatrick Hunt 2012-11-27, 17:42
On Tue, Nov 27, 2012 at 4:40 AM, Brian Tarbox <[EMAIL PROTECTED]> wrote:
> Does this warning mean a: the log is too big, b: zk is competing with another process for disk access, c: other?
>
That's timing how long the fsync takes on the txnlog, typically it
runs log because the disk is busy, or the OS has a large number of
dirty pages on the volume (ext3 esp), etc...
> How should one respond to the warning?
The admin and troubleshooting guides both provide some insight, but in
particular the txnlog should be on a dedicated spindle (non-vm obv) if
you care about performance (latency). Any fsync timing that's large
than/near your session timeout is going to be trouble.
Patrick
> On Nov 27, 2012, at 2:47 AM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>
>> On Mon, Nov 26, 2012 at 11:43 PM, Narayanan A R
>> <[EMAIL PROTECTED]> wrote:
>>> Have you seen this before?
>>>
>>> 2012-11-25 16:01:08,074 [myid:1] - WARN [SyncThread:1:FileTxnLog@321] -
>>> fsync-ing the write ahead log in SyncThread:1 took 1470ms which will
>>> adversely effect operation latency. See the ZooKeeper troubleshooting guide
>>>
>>> I am running ZK cluster of size 3 in a VM.
>>
>> Yes, I added it recently to highlight situations where environment
>> could adversely effect ZK latency.
>>
>> Patrick
> Does this warning mean a: the log is too big, b: zk is competing with another process for disk access, c: other?
>
That's timing how long the fsync takes on the txnlog, typically it
runs log because the disk is busy, or the OS has a large number of
dirty pages on the volume (ext3 esp), etc...
> How should one respond to the warning?
The admin and troubleshooting guides both provide some insight, but in
particular the txnlog should be on a dedicated spindle (non-vm obv) if
you care about performance (latency). Any fsync timing that's large
than/near your session timeout is going to be trouble.
Patrick
> On Nov 27, 2012, at 2:47 AM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>
>> On Mon, Nov 26, 2012 at 11:43 PM, Narayanan A R
>> <[EMAIL PROTECTED]> wrote:
>>> Have you seen this before?
>>>
>>> 2012-11-25 16:01:08,074 [myid:1] - WARN [SyncThread:1:FileTxnLog@321] -
>>> fsync-ing the write ahead log in SyncThread:1 took 1470ms which will
>>> adversely effect operation latency. See the ZooKeeper troubleshooting guide
>>>
>>> I am running ZK cluster of size 3 in a VM.
>>
>> Yes, I added it recently to highlight situations where environment
>> could adversely effect ZK latency.
>>
>> Patrick
