Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Balancing a cluster when a new node is added


Copy link to this message
-
Re: Balancing a cluster when a new node is added
I think that port itself is blocked. I'll contact the sysadmins.
Thanks

On Mon, Jan 11, 2010 at 7:46 PM, Saptarshi Guha
<[EMAIL PROTECTED]> wrote:
>> What happens if you give the balancer command a threshold?
>>
>>
> So I gave a threshold, (0.20) and it started to run and I got several
> errors like this
>
> 10/01/11 19:43:56 WARN balancer.Balancer: Error moving block
> 795170313073485718 from spica:50010 to altair:50010 through
> 128.210.141.89:50010: No route to host
>
> (altair is the node i added).
> I don't know why there isn't a route to the host, since I can start
> the node automatically(via ssh), as seen below the report shows it to
> be there. Is  a no route to host possible if the that particular 50010
> port is closed?
>
>>
>>> Q2. On the DFS UI website, when I click on the existing nodes to see
>>> data, I can, but when I click on the new node, i can't connect.
>>> Does this happen when there are no files? The datanode log for this
>>> machine does not show any errors. I have managed to copy a small file
>>> this new machine (from the new machine, so the file is stored on this
>>> machines section of the DFS)
>>
>>
>> Does the namenode actually recognize the new node?  What does dfsadmin
>> -report tell you?
>
> The report shows it to be present,
> Name: A.B.C.D:50010
> Decommission Status : Normal
> Configured Capacity: 1056894091264 (984.31 GB)
> DFS Used: 524288 (512 KB)
> Non DFS Used: 55336439808 (51.54 GB)
> DFS Remaining: 1001557127168(932.77 GB)
> DFS Used%: 0%
> DFS Remaining%: 94.76%
> Last contact: Mon Jan 11 19:40:35 EST 2010
>
>> Are you using a dfs.hosts (aka include) file?  Is it
>> listed?  Are you using a dfs.hosts.exclude file?  Is it listed there on
>> accident?
>>
> No dfs.hosts, nor excludes. I stopped the cluster (stop-dfs.sh) added
> the machine(called altair) to the cluster(in the slaves file) and
> bought it back up.
>
>
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB