Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> HDFS append on upgraded cluster


+
Vinayakumar B 2012-11-16, 07:00
Copy link to this message
-
Re: HDFS append on upgraded cluster
Hi Vinayakumar,

If you append to a block which has hard-links, then it will get copied on
write at the DataNode. See this code:

  /**
   * Make a copy of the block if this block is linked to an existing
   * snapshot. This ensures that modifying this block does not modify
   * data in any existing snapshots.
   * @param block Block
   * @param numLinks Detach if the number of links exceed this value
   * @throws IOException
   * @return - true if the specified block was detached
   */
  public boolean detachBlock(Block block, int numLinks) throws IOException {
...

-Todd

On Thu, Nov 15, 2012 at 11:00 PM, Vinayakumar B <[EMAIL PROTECTED]>wrote:

> Hi All,
>
>
>
> I have a question here.
>
>
>
> Scenario:
>
> --------------
>
> 1.       Cluster is having some blocks and its upgraded.
>
> 2.       As part of upgrade, in datanode side hardlinks are created for all
> blocks.
>
> 3.       Now one of the block is appended with the some more data
>
> 4.       Do rollback
>
>
>
> Now what happens for the appended data..?
>
>
>
> Since blocks are hardlinked, changes on the original file will reflect on
> the hardlinked file inside previous  also.
>
>
>
> So changes to a appended block after upgrade will not be reverted after
> rollback.
>
>
>
> How to handle this case..?
>
>
>
> Any thoughts..?
>
>
>
> Regards,
>
>    Vinayakumar B.,
>
>
> ****************************************************************************
> This e-mail and attachments contain confidential information from HUAWEI,
> which is intended only for the person or entity whose address is listed
> above. Any use of the information contained herein in any way (including,
> but not limited to, total or partial disclosure, reproduction, or
> dissemination) by persons other than the intended recipient's) is
> prohibited. If you receive this e-mail in error, please notify the sender
> by
> phone or email immediately and delete it!
>
>
>
>
--
Todd Lipcon
Software Engineer, Cloudera