Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: Rack-awareness in Hadoop-2.0.3-alpha


Copy link to this message
-
Re: Rack-awareness in Hadoop-2.0.3-alpha
Patai Sangbutsarakum 2013-05-22, 17:55
I believe that his topology.sh and .data files are already correct.
bash topology.sh 172.31.13.133 mustaqeem-1 mustaqeem-4
/default/rack /rack2 /rack3

output looks exactly the same as mine.
Mohammad,
1. did you restart namenode after you modified the configuration? in 0.20,
restart namenode is required.
2. Try to revert the scheduler to the default i believe it is FIFO, and
play with Rack-awareness again.
3. Check if you need to chown topology.sh to the user that run namenode
4. Check if you need to chmod u+x topology.sh
5. Check if topology.data can be read by the user.
hope this help

On Wed, May 22, 2013 at 9:49 AM, Chris Nauroth <[EMAIL PROTECTED]>wrote:

> common-dev and hdfs-dev removed/bcc'd
>
> Hi Mohammad,
>
> Rack awareness is supported in 2.0.3-alpha.  The only potential problem I
> see in your configuration is that topology.sh contains a definition for
> HADOOP_CONF that points back at your hadoop-0.22.0/conf directory.  If that
> directory doesn't contain the right topology.data file, then resolution to
> a rack might not work as you expected.
>
> If that doesn't fix the problem, then I have another suggestion: change
> topology.sh to add some debugging statements that echo to a temp file the
> arguments that the script has received, and the results it decides to
> print.  This would show you exactly how Hadoop called your script and
> exactly what rack your script replied back to Hadoop.  It might give you an
> idea for what to investigate next.
>
> Hope this helps,
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Wed, May 22, 2013 at 6:29 AM, Mohammad Mustaqeem <
> [EMAIL PROTECTED]> wrote:
>
>> Is Hadoop-2.0.3-alpha does not support Rack-awarness?
>> I am trying to make Hadoop cluster Rack-Aware for a week but I
>> haven't succeed.
>>
>> What I am doing.
>> I am adding following property in etc/hadoop/core-site.xml :
>> <property>
>> <name>net.topology.script.file.name</name>
>> <value>/home/hadoop/hadoop-2.0.3-alpha/etc/hadoop/topology.sh</value>
>> </property>
>> The topology.sh and topology.data is attached with this mail.
>>
>> I want to tell you that once I used this property(
>> topology.script.file.name) and files to make Hadoop-0.22.0 rack-aware
>> but I am unable to do the same for Hadoop-2.0.3-alpha.
>> What I am doing wrong?
>> Please, somebody help me.
>>
>>
>> --
>> *With regards ---*
>> *Mohammad Mustaqeem*,
>> M.Tech (CSE)
>> MNNIT Allahabad
>>
>
>