Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Testing rack awareness


Copy link to this message
-
RE: Testing rack awareness
"rack awareness" is a bit of a misnomer.  Hadoop is not aware of racks in the sense that it can figure out your network topology.  Instead, you need to write a script that tells Hadoop which machines belong to which racks.  For example:  http://wiki.apache.org/hadoop/topology_rack_awareness_scripts.  You can tell if your script is working when "default-rack" is replaced by one of your rack names.

Jeff

From: Jain, Prem [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, February 01, 2012 1:55 PM
To: [EMAIL PROTECTED]
Subject: Testing rack awareness

Is there a way to check if the rack awareness is working ?
I seem to configure it right, but  when I run a fsck on my files written, it still shows  "Number of racks:               1"
What I am looking for is a simple command which can tell me if rack awareness is working or not ?
Thanks in advance

------------------------------------------------------------------------------------------------------------------

root@namenode]# hadoop fsck /user/file  -files -blocks -locations -racks
FSCK started by root from /10.61.97.224 for path /user/file  at Wed Feb 01 14:36:42 EST 2012
/user/file  bytes, 1 block(s):  OK
0. blk_-2693119624313554881_407726 len=18483674 repl=2 [/default-rack/10.61.97.214:50010, /default-rack/10.61.97.217:50010]

Status: HEALTHY
Total size:    18483674 B
Total dirs:    0
Total files:   1
Total blocks (validated):      1 (avg. block size 18483674 B)
Minimally replicated blocks:   1 (100.0 %)
Over-replicated blocks:        0 (0.0 %)
Under-replicated blocks:       0 (0.0 %)
Mis-replicated blocks:         0 (0.0 %)
Default replication factor:    2
Average block replication:     2.0
Corrupt blocks:                0
Missing replicas:              0 (0.0 %)
Number of data-nodes:          8
Number of racks:               1
FSCK ended at Wed Feb 01 14:36:42 EST 2012 in 0 milliseconds
The filesystem under path '/user/file  ' is HEALTHY

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB