Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Testing rack awareness


Copy link to this message
-
RE: Testing rack awareness
Jeffrey Buell 2012-02-01, 22:10
"rack awareness" is a bit of a misnomer.  Hadoop is not aware of racks in the sense that it can figure out your network topology.  Instead, you need to write a script that tells Hadoop which machines belong to which racks.  For example:  http://wiki.apache.org/hadoop/topology_rack_awareness_scripts.  You can tell if your script is working when "default-rack" is replaced by one of your rack names.

Jeff

From: Jain, Prem [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, February 01, 2012 1:55 PM
To: [EMAIL PROTECTED]
Subject: Testing rack awareness

Is there a way to check if the rack awareness is working ?
I seem to configure it right, but  when I run a fsck on my files written, it still shows  "Number of racks:               1"
What I am looking for is a simple command which can tell me if rack awareness is working or not ?
Thanks in advance

------------------------------------------------------------------------------------------------------------------

root@namenode]# hadoop fsck /user/file  -files -blocks -locations -racks
FSCK started by root from /10.61.97.224 for path /user/file  at Wed Feb 01 14:36:42 EST 2012
/user/file  bytes, 1 block(s):  OK
0. blk_-2693119624313554881_407726 len=18483674 repl=2 [/default-rack/10.61.97.214:50010, /default-rack/10.61.97.217:50010]

Status: HEALTHY
Total size:    18483674 B
Total dirs:    0
Total files:   1
Total blocks (validated):      1 (avg. block size 18483674 B)
Minimally replicated blocks:   1 (100.0 %)
Over-replicated blocks:        0 (0.0 %)
Under-replicated blocks:       0 (0.0 %)
Mis-replicated blocks:         0 (0.0 %)
Default replication factor:    2
Average block replication:     2.0
Corrupt blocks:                0
Missing replicas:              0 (0.0 %)
Number of data-nodes:          8
Number of racks:               1
FSCK ended at Wed Feb 01 14:36:42 EST 2012 in 0 milliseconds
The filesystem under path '/user/file  ' is HEALTHY