Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Is it safe enough to write edit log in buffer before writing to disk ?


Copy link to this message
-
Is it safe enough to write edit log in buffer before writing to disk ?
Hi all,

I check the source code of EditLogFileOutputStream, it seems hadoop
will first write edit log to buffer, then flush to disk. I know that
it will improve performance, but in the other hand it will cause the
edit log in buffer lost when the name node is down. So I wonder is it
possible and necessary to let users do configuration in the conf file
to decide whether use buffer or not? Thanks.

--
Best Regards

Jeff Zhang
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB