-Re: Balancing a cluster when a new node is added
Saptarshi Guha 2010-01-12, 00:46
> What happens if you give the balancer command a threshold?
So I gave a threshold, (0.20) and it started to run and I got several
errors like this
10/01/11 19:43:56 WARN balancer.Balancer: Error moving block
795170313073485718 from spica:50010 to altair:50010 through
22.214.171.124:50010: No route to host
(altair is the node i added).
I don't know why there isn't a route to the host, since I can start
the node automatically(via ssh), as seen below the report shows it to
be there. Is a no route to host possible if the that particular 50010
port is closed?
>> Q2. On the DFS UI website, when I click on the existing nodes to see
>> data, I can, but when I click on the new node, i can't connect.
>> Does this happen when there are no files? The datanode log for this
>> machine does not show any errors. I have managed to copy a small file
>> this new machine (from the new machine, so the file is stored on this
>> machines section of the DFS)
> Does the namenode actually recognize the new node? What does dfsadmin
> -report tell you?
The report shows it to be present,
Decommission Status : Normal
Configured Capacity: 1056894091264 (984.31 GB)
DFS Used: 524288 (512 KB)
Non DFS Used: 55336439808 (51.54 GB)
DFS Remaining: 1001557127168(932.77 GB)
DFS Used%: 0%
DFS Remaining%: 94.76%
Last contact: Mon Jan 11 19:40:35 EST 2010
> Are you using a dfs.hosts (aka include) file? Is it
> listed? Are you using a dfs.hosts.exclude file? Is it listed there on
No dfs.hosts, nor excludes. I stopped the cluster (stop-dfs.sh) added
the machine(called altair) to the cluster(in the slaves file) and
bought it back up.