Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: Block placement Policy


Copy link to this message
-
RE: Block placement Policy
Ivan Mitic 2013-05-04, 16:10
Hi Mohammad,

You will have to implement BlockPlacementPolicy interface (abstract class). There are two existing implementations in Hadoop trunk, BlockPlacementPolicyDefault and BlockPlacementPolicyWithNodeGroup.

To configure Hadoop to use your policy, define dfs.block.replicator.classname in hdfs-site.xml. For example:

  <property>
    <name>dfs.block.replicator.classname</name>
    <value>org.apache.hadoop.hdfs.server.namenode.AzureBlockPlacementPolicy</value>
  </property>

Usually, you will also want to have your own script that computes the node topology/location (look for net.topology.script.file.name in trunk, or topology.script.file.name in branch-1).

Hope this helps,
Ivan

From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
Sent: Saturday, May 04, 2013 8:35 AM
To: [EMAIL PROTECTED]
Subject: Re: Block placement Policy

You might find this useful : https://issues.apache.org/jira/browse/HDFS-385

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com<http://cloudfront.blogspot.com>

On Sat, May 4, 2013 at 8:57 PM, Mohammad Mustaqeem <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
I have read somewhere that a user can specified his own ReplicaPlacementPolicy.
How can I specify my own ReplicaPlacementPolicy?
I you have any sample ReplicaPlacementPolicy, then please share it..

--
*With regards ---*
*Mohammad Mustaqeem*,
M.Tech (CSE)
MNNIT Allahabad
9026604270