Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Balancing a cluster when a new node is added


Copy link to this message
-
Re: Balancing a cluster when a new node is added
I think that port itself is blocked. I'll contact the sysadmins.
Thanks

On Mon, Jan 11, 2010 at 7:46 PM, Saptarshi Guha
<[EMAIL PROTECTED]> wrote:
>> What happens if you give the balancer command a threshold?
>>
>>
> So I gave a threshold, (0.20) and it started to run and I got several
> errors like this
>
> 10/01/11 19:43:56 WARN balancer.Balancer: Error moving block
> 795170313073485718 from spica:50010 to altair:50010 through
> 128.210.141.89:50010: No route to host
>
> (altair is the node i added).
> I don't know why there isn't a route to the host, since I can start
> the node automatically(via ssh), as seen below the report shows it to
> be there. Is  a no route to host possible if the that particular 50010
> port is closed?
>
>>
>>> Q2. On the DFS UI website, when I click on the existing nodes to see
>>> data, I can, but when I click on the new node, i can't connect.
>>> Does this happen when there are no files? The datanode log for this
>>> machine does not show any errors. I have managed to copy a small file
>>> this new machine (from the new machine, so the file is stored on this
>>> machines section of the DFS)
>>
>>
>> Does the namenode actually recognize the new node?  What does dfsadmin
>> -report tell you?
>
> The report shows it to be present,
> Name: A.B.C.D:50010
> Decommission Status : Normal
> Configured Capacity: 1056894091264 (984.31 GB)
> DFS Used: 524288 (512 KB)
> Non DFS Used: 55336439808 (51.54 GB)
> DFS Remaining: 1001557127168(932.77 GB)
> DFS Used%: 0%
> DFS Remaining%: 94.76%
> Last contact: Mon Jan 11 19:40:35 EST 2010
>
>> Are you using a dfs.hosts (aka include) file?  Is it
>> listed?  Are you using a dfs.hosts.exclude file?  Is it listed there on
>> accident?
>>
> No dfs.hosts, nor excludes. I stopped the cluster (stop-dfs.sh) added
> the machine(called altair) to the cluster(in the slaves file) and
> bought it back up.
>
>
>>
>>
>