Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: Replication Factor Modification


Copy link to this message
-
Re: Replication Factor Modification
Hi  Uddipan

As Harsh mentioned, replication factor is a client side property . So you
need to update the value for 'dfs.replication' in hdfs-site.xml as per your
requirement in your edge nodes or from the machines your are copying files
to hdfs. If you are using some of the existing DN's for this purpose (as
client) you need to update the value in there. No need of restarting the
services.

On Wed, Sep 5, 2012 at 11:54 PM, Uddipan Mukherjee <
[EMAIL PROTECTED]> wrote:

>  Hi,****
>
> ** **
>
>    Thanks for the help. But How I will set the replication factor as
> desired so that when new files comes in it will automatically take the new
> value of dfs.replication without a cluster restart. Please note we have a
> 200 nodes cluster.****
>
> ** **
>
> Thanks and Regards,****
>
> Uddipan Mukherjee****
>
> ** **
>
> *From:* Harsh J [mailto:[EMAIL PROTECTED]]
> *Sent:* Wednesday, September 05, 2012 7:17 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Replication Factor Modification****
>
> ** **
>
> Replication factor is per-file, and is a client-side property. So, this is
> doable.****
>
> ** **
>
> 1. Change the replication factor of all existing files (or needed ones):**
> **
>
> ** **
>
> $ hadoop fs -setrep -R <value> /****
>
> ** **
>
> 2. Change the dfs.replication parameter in all client configs to the
> desired <value>****
>
> On Wed, Sep 5, 2012 at 11:39 PM, Uddipan Mukherjee <
> [EMAIL PROTECTED]> wrote:****
>
> Hi, ****
>
>  ****
>
>    We have a requirement where we have change our Hadoop Cluster's
> Replication Factor without restarting the Cluster. We are running our
> Cluster on Amazon EMR.****
>
>  ****
>
> Can you please suggest the way to achieve this? Any pointer to this will
> be very helpful.****
>
>  ****
>
> Thanks And Regards****
>
> Uddipan Mukherjee****
>
> **************** CAUTION - Disclaimer *********************
>
> This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely ****
>
> for the use of the addressee(s). If you are not the intended recipient, please ****
>
> notify the sender by e-mail and delete the original message. Further, you are not ****
>
> to copy, disclose, or distribute this e-mail or its contents to any other person and ****
>
> any such actions are unlawful. This e-mail may contain viruses. Infosys has taken ****
>
> every reasonable precaution to minimize this risk, but is not liable for any damage ****
>
> you may sustain as a result of any virus in this e-mail. You should carry out your ****
>
> own virus checks before opening the e-mail or attachment. Infosys reserves the ****
>
> right to monitor and review the content of all messages sent to or from this e-mail ****
>
> address. Messages sent to or from this e-mail address may be stored on the ****
>
> Infosys e-mail system.****
>
> ***INFOSYS******** End of Disclaimer ********INFOSYS*******
>
>
>
> ****
>
> ** **
>
> --
> Harsh J****
>
+
Harsh J 2012-09-05, 18:17
+
Uma Maheswara Rao G 2012-09-05, 18:16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB