Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Balancing a cluster when a new node is added


Copy link to this message
-
Re: Balancing a cluster when a new node is added
> What happens if you give the balancer command a threshold?
>
>
So I gave a threshold, (0.20) and it started to run and I got several
errors like this

10/01/11 19:43:56 WARN balancer.Balancer: Error moving block
795170313073485718 from spica:50010 to altair:50010 through
128.210.141.89:50010: No route to host

(altair is the node i added).
I don't know why there isn't a route to the host, since I can start
the node automatically(via ssh), as seen below the report shows it to
be there. Is  a no route to host possible if the that particular 50010
port is closed?

>
>> Q2. On the DFS UI website, when I click on the existing nodes to see
>> data, I can, but when I click on the new node, i can't connect.
>> Does this happen when there are no files? The datanode log for this
>> machine does not show any errors. I have managed to copy a small file
>> this new machine (from the new machine, so the file is stored on this
>> machines section of the DFS)
>
>
> Does the namenode actually recognize the new node?  What does dfsadmin
> -report tell you?

The report shows it to be present,
Name: A.B.C.D:50010
Decommission Status : Normal
Configured Capacity: 1056894091264 (984.31 GB)
DFS Used: 524288 (512 KB)
Non DFS Used: 55336439808 (51.54 GB)
DFS Remaining: 1001557127168(932.77 GB)
DFS Used%: 0%
DFS Remaining%: 94.76%
Last contact: Mon Jan 11 19:40:35 EST 2010

> Are you using a dfs.hosts (aka include) file?  Is it
> listed?  Are you using a dfs.hosts.exclude file?  Is it listed there on
> accident?
>
No dfs.hosts, nor excludes. I stopped the cluster (stop-dfs.sh) added
the machine(called altair) to the cluster(in the slaves file) and
bought it back up.
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB