Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # dev - Is it safe enough to write edit log in buffer before writing to disk ?


Copy link to this message
-
Is it safe enough to write edit log in buffer before writing to disk ?
Jeff Zhang 2010-06-11, 01:29
Hi all,

I check the source code of EditLogFileOutputStream, it seems hadoop
will first write edit log to buffer, then flush to disk. I know that
it will improve performance, but in the other hand it will cause the
edit log in buffer lost when the name node is down. So I wonder is it
possible and necessary to let users do configuration in the conf file
to decide whether use buffer or not? Thanks.

--
Best Regards

Jeff Zhang