Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Calling o/s.flush() in HLog.sync()?


+
Himanshu Vashishtha 2013-11-05, 05:11
+
Haosong Huang 2013-11-05, 09:08
Copy link to this message
-
Re: Calling o/s.flush() in HLog.sync()?
Okay, good to know but not sure how your response is related to what I
asked.
On Tue, Nov 5, 2013 at 1:08 AM, Haosong Huang <[EMAIL PROTECTED]> wrote:

> An os fsync() call will spent nearly 10ms because of the harddisk iops
> neckbottle. A hsync() would become two os fsync(). One for checksum file
> and theother for block file. If you use SSD disk, you could try use fsync()
> instead of flush() and mount file system without writebarrier.
> 2013-11-5 下午1:12于 "Himanshu Vashishtha" <[EMAIL PROTECTED]>写道:
>
> > Looking at ProtobufLogWriter class, it looks like the call to flush() in
> > the sync method is a noop.
> >
> >
> >
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L134
> >
> > The underlying output stream is DFSOutputStream, which doesn't implement
> > flush().
> >
> > And, it calls sync() anyway, which ensures the data is written to DN's
> > (cache).
> >
> > Previously with SequenceFile$Writer, it writes data to the outputstream
> > (using Writables#write), and invoke sync/hflush.
> >
> >
> https://github.com/apache/hadoop-common/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java#L1314
> >
> > Is there a reason we have this call here? Please let me know if I miss
> any
> > context.
> >
> > Thanks,
> > Himanshu
> >
>
+
Haosong Huang 2013-11-08, 14:57
+
Ted Yu 2013-11-07, 22:56
+
Jonathan Hsieh 2013-11-15, 07:37
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB