Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # dev - Review Request: FLUME-1699. Make the rename of the meta file platform neutral


Copy link to this message
-
Re: Review Request: FLUME-1699. Make the rename of the meta file platform neutral
Hari Shreedharan 2012-11-16, 20:08


> On Nov. 16, 2012, 7:48 p.m., Roshan Naik wrote:
> > why not simplify it to ..
> >
> >
> > //  delete log.meta  
> > //  log.meta.tmp -> log.meta
> >

I am not entirely sure how all platforms handle renames. Depending on implementation, though highly unlikely, if the system crashes in the middle of a rename, the tmp file might disappear without the ".meta" file being created. Leaving the old file guarantees that either the old or the new file is available at any point in time.

I am just being paranoid about it since I don't know how different operating systems implement renames (for example, Windows does not seem to give many guarantees about durability in the event of a crash during a rename)
- Hari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8087/#review13525
-----------------------------------------------------------
On Nov. 16, 2012, 7:26 p.m., Hari Shreedharan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8087/
> -----------------------------------------------------------
>
> (Updated Nov. 16, 2012, 7:26 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> If atomic rename fails, rename the current meta file to old, rename tmp to meta and then delete old.
>
>
> This addresses bug FLUME-1699.
>     https://issues.apache.org/jira/browse/FLUME-1699
>
>
> Diffs
> -----
>
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFileV3.java f768d23
>
> Diff: https://reviews.apache.org/r/8087/diff/
>
>
> Testing
> -------
>
> All unit tests pass
>
>
> Thanks,
>
> Hari Shreedharan
>
>