Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Rack Aware Hadoop cluster


+
Mohammad Mustaqeem 2013-05-08, 13:43
+
Chris Embree 2013-05-08, 13:52
+
Shahab Yunus 2013-05-08, 13:52
+
Mohammad Mustaqeem 2013-05-08, 14:01
Copy link to this message
-
Re: Rack Aware Hadoop cluster
Chris Embree 2013-05-08, 14:11
Your script has an error in it.  Please test your script using both IP
Addresses and Names, outside of hadoop.
On Wed, May 8, 2013 at 10:01 AM, Mohammad Mustaqeem
<[EMAIL PROTECTED]>wrote:

> I have done this and found following error in log -
>
> 2013-05-08 18:53:45,221 WARN org.apache.hadoop.net.ScriptBasedMapping: Exception running /home/mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rack.sh 127.0.0.1
> org.apache.hadoop.util.Shell$ExitCodeException: /home/mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rack.sh: 8: /home/mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rack.sh: Syntax error: "(" unexpected (expecting "done")
>
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:202)
> at org.apache.hadoop.util.Shell.run(Shell.java:129)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:322)
> at org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.runResolveCommand(ScriptBasedMapping.java:241)
> at org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.resolve(ScriptBasedMapping.java:179)
> at org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:119)
> at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.resolveNetworkLocation(DatanodeManager.java:454)
> at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:713)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:3459)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:881)
> at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:90)
> at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:18295)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:454)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1014)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1735)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1731)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1441)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1729)
> 2013-05-08 18:53:45,223 ERROR org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: The resolve call returned null! Using /default-rack for host [127.0.0.1]
>
>
>
> On Wed, May 8, 2013 at 7:18 PM, Leonid Fedotov <[EMAIL PROTECTED]>wrote:
>
>> You can put this parameter to core-site.xml or hdfs-site.xml
>> It both parsed during the HDFS startup.
>>
>> Leonid
>>
>>
>> On Wed, May 8, 2013 at 6:43 AM, Mohammad Mustaqeem <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hello everyone,
>>>     I was searching for how to make the hadoop cluster rack-aware and I
>>> find out from here
>>> http://hadoop.apache.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/ClusterSetup.html#Hadoop_Rack_Awareness that
>>> we can do this by giving property of "topology.script.file.name". But
>>> here it is not written where to put this
>>> <property>
>>>         <name>topology.script.file.name</name>
>>>
>>> <value>/home/mustaqeem/development/hadoop-2.0.3-alpha/etc/hadoop/rack.sh</value>
>>> </property>
>>>
>>> Means in which configuration file.
>>> I am using hadoop-2.0.3-alpha.
>>>
>>>
>>> --
>>> *With regards ---*
>>> *Mohammad Mustaqeem*,
>>> M.Tech (CSE)
>>> MNNIT Allahabad
>>> 9026604270
>>>
>>>
>>>
>>
>
>
> --
> *With regards ---*
> *Mohammad Mustaqeem*,
> M.Tech (CSE)
> MNNIT Allahabad
> 9026604270
>
>
>
+
Leonid Fedotov 2013-05-08, 14:18
+
Mohammad Mustaqeem 2013-05-08, 17:00
+
Mohammad Mustaqeem 2013-05-08, 17:29
+
Adam Faris 2013-05-08, 17:38
+
Chris Embree 2013-05-08, 20:49
+
Serge Blazhievsky 2013-05-08, 20:52
+
Mohammad Mustaqeem 2013-05-09, 10:07
+
Mohammad Mustaqeem 2013-05-09, 16:30
+
Mohammad Mustaqeem 2013-05-09, 19:00