haosdent 2013-08-25, 05:11
50ms is believable. hsync makes each DN call fsync and wait for acks, so
you'd expect at least a disk seek time (~10ms) with some extra time
depending on how much unsync'd data is being written.
So, just as some back of the envelope math, assuming a disk that can write
50ms - 10ms seek = 40ms writing time
100 MB/s * 40ms = 4MB
If you're hsync'ing every 4MB, 50ms would be exactly what I'd expect.
On Sat, Aug 24, 2013 at 10:11 PM, haosdent <[EMAIL PROTECTED]> wrote:
> Hi, all. Hadoop support hsync which would call fsync of system after
> 2.0.2. I have tested the performance of hsync() and hflush() again and
> again, but I found that the hsync call() everytime would spent nearly 50ms
> while the hflush call() just spent 2ms. In this slide(
> http://www.slideshare.net/enissoz/hbase-and-hdfs-understanding-filesystem-usagePage 18), the author mentions that hsync() is 2x slower than hflush(). So,
> is anything wrong? Thank you very much and looking forward to your help.
> Best Regards,
> Haosong Huang
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
haosdent 2013-08-26, 02:44
Andrew Wang 2013-08-26, 03:18
haosdent 2013-08-26, 03:21
lei liu 2013-08-26, 14:30
Andrew Wang 2013-08-26, 17:44