Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> HDFS append on upgraded cluster


Copy link to this message
-
Re: HDFS append on upgraded cluster
Hi Vinayakumar,

If you append to a block which has hard-links, then it will get copied on
write at the DataNode. See this code:

  /**
   * Make a copy of the block if this block is linked to an existing
   * snapshot. This ensures that modifying this block does not modify
   * data in any existing snapshots.
   * @param block Block
   * @param numLinks Detach if the number of links exceed this value
   * @throws IOException
   * @return - true if the specified block was detached
   */
  public boolean detachBlock(Block block, int numLinks) throws IOException {
...

-Todd

On Thu, Nov 15, 2012 at 11:00 PM, Vinayakumar B <[EMAIL PROTECTED]>wrote:

> Hi All,
>
>
>
> I have a question here.
>
>
>
> Scenario:
>
> --------------
>
> 1.       Cluster is having some blocks and its upgraded.
>
> 2.       As part of upgrade, in datanode side hardlinks are created for all
> blocks.
>
> 3.       Now one of the block is appended with the some more data
>
> 4.       Do rollback
>
>
>
> Now what happens for the appended data..?
>
>
>
> Since blocks are hardlinked, changes on the original file will reflect on
> the hardlinked file inside previous  also.
>
>
>
> So changes to a appended block after upgrade will not be reverted after
> rollback.
>
>
>
> How to handle this case..?
>
>
>
> Any thoughts..?
>
>
>
> Regards,
>
>    Vinayakumar B.,
>
>
> ****************************************************************************
> This e-mail and attachments contain confidential information from HUAWEI,
> which is intended only for the person or entity whose address is listed
> above. Any use of the information contained herein in any way (including,
> but not limited to, total or partial disclosure, reproduction, or
> dissemination) by persons other than the intended recipient's) is
> prohibited. If you receive this e-mail in error, please notify the sender
> by
> phone or email immediately and delete it!
>
>
>
>
--
Todd Lipcon
Software Engineer, Cloudera
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB