Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Is it safe enough to write edit log in buffer before writing to disk ?


Copy link to this message
-
Re: Is it safe enough to write edit log in buffer before writing to disk ?
Thanks, Todd. I saw the flush code.

On Fri, Jun 11, 2010 at 9:31 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> Hi Jeff,
> All of the FSNamesystem methods call logSync() before returning to the
> client. So, if the edit is lost, it also will not have returned a success to
> the client.
> -Todd
>
> On Thu, Jun 10, 2010 at 6:29 PM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>>
>> Hi all,
>>
>> I check the source code of EditLogFileOutputStream, it seems hadoop
>> will first write edit log to buffer, then flush to disk. I know that
>> it will improve performance, but in the other hand it will cause the
>> edit log in buffer lost when the name node is down. So I wonder is it
>> possible and necessary to let users do configuration in the conf file
>> to decide whether use buffer or not? Thanks.
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

--
Best Regards

Jeff Zhang
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB