Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Calling o/s.flush() in HLog.sync()?


Copy link to this message
-
Re: Calling o/s.flush() in HLog.sync()?
Okay, good to know but not sure how your response is related to what I
asked.
On Tue, Nov 5, 2013 at 1:08 AM, Haosong Huang <[EMAIL PROTECTED]> wrote:

> An os fsync() call will spent nearly 10ms because of the harddisk iops
> neckbottle. A hsync() would become two os fsync(). One for checksum file
> and theother for block file. If you use SSD disk, you could try use fsync()
> instead of flush() and mount file system without writebarrier.
> 2013-11-5 下午1:12于 "Himanshu Vashishtha" <[EMAIL PROTECTED]>写道:
>
> > Looking at ProtobufLogWriter class, it looks like the call to flush() in
> > the sync method is a noop.
> >
> >
> >
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L134
> >
> > The underlying output stream is DFSOutputStream, which doesn't implement
> > flush().
> >
> > And, it calls sync() anyway, which ensures the data is written to DN's
> > (cache).
> >
> > Previously with SequenceFile$Writer, it writes data to the outputstream
> > (using Writables#write), and invoke sync/hflush.
> >
> >
> https://github.com/apache/hadoop-common/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java#L1314
> >
> > Is there a reason we have this call here? Please let me know if I miss
> any
> > context.
> >
> > Thanks,
> > Himanshu
> >
>