Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # dev - Regarding hsync


+
Hemant Bhanawat 2013-07-09, 07:25
+
Hemant Bhanawat 2013-07-11, 08:38
Copy link to this message
-
Re: Regarding hsync
Jing Zhao 2013-07-11, 17:12
Hi Hemant,

    HDFS-4213 (https://issues.apache.org/jira/browse/HDFS-4213) may be
the one you're looking for. In general, an hsync which can update the
file size on NN is enabled. You may call
"hsync(EnumSet.of(SyncFlag.UPDATE_LENGTH))" to do that.

Thanks,
-Jing

On Thu, Jul 11, 2013 at 1:38 AM, Hemant Bhanawat <[EMAIL PROTECTED]> wrote:
> Hi,
>
> Any help?
>
> Thanks in advance,
> Hemant
>
> ----- Original Message -----
>
> From: "Hemant Bhanawat" <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Tuesday, July 9, 2013 12:55:23 PM
> Subject: Regarding hsync
>
> Hi,
>
> I am currently working on hadoop version 2.0.*.
>
> Currently, hsync does not update the file size on namenode. So, if my process dies after calling hsync but before calling file close, the file is left with an inconsistent file size. I would like to fix this file size. Is there a way to do that? A workaround that I have come across is to open the file stream in append mode and close it. This fixes the file size on the namenode. Is it a reliable solution?
>
> Thanks,
> Hemant
>