|
|
-
No live nodes on hdfs clusterRicardo Casazza 2013-02-08, 16:58
Hi!
I've followed the hadoop cluster tutorial on hadoop site (hadoop 1.1.1 on 64bit machines with openjdk 1.6). I've set-up 1 namenode, 1 jobtracker, and 3 slaves acting as datanode and tasktracker. I have a problem setting up hdfs on the cluster: dfs daemon start fine on namenode and datanodes but when I go to http://namenode:50070/ I have this: Configured Capacity : 0 KB DFS Used : 0 KB Non DFS Used : 0 KB DFS Remaining : 0 KB DFS Used% : 100 % DFS Remaining% : 0 % Live Nodes : 0 Dead Nodes : 0 Decommissioning Nodes : 0 Number of Under-Replicated Blocks : 0 I've read disk space could be a problem, but i've checked there are 10GB of free space on each datanode. Here the logs of one of the datanodes: /************************************************************ STARTUP_MSG: Starting DataNode STARTUP_MSG: host = ncepspa117/172.16.140.117 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.1.1 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1 -r 1411108; compiled by 'hortonfo' on Mon Nov 19 10:48:11 UTC 2012 ************************************************************/ 2013-02-08 16:36:49,881 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2013-02-08 16:36:49,892 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2013-02-08 16:36:49,892 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2013-02-08 16:36:49,893 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started 2013-02-08 16:36:49,985 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2013-02-08 16:36:50,328 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Registered FSDatasetStatusMBean 2013-02-08 16:36:50,340 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened data transfer server at 50010 2013-02-08 16:36:50,343 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 1048576 bytes/s 2013-02-08 16:36:50,388 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2013-02-08 16:36:50,448 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2013-02-08 16:36:50,459 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dfs.webhdfs.enabled = false 2013-02-08 16:36:50,460 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50075 2013-02-08 16:36:50,460 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50075 webServer.getConnectors()[0].getLocalPort() returned 50075 2013-02-08 16:36:50,460 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50075 2013-02-08 16:36:50,460 INFO org.mortbay.log: jetty-6.1.26 2013-02-08 16:36:50,729 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:50075 2013-02-08 16:36:50,735 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered. 2013-02-08 16:36:50,735 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source DataNode registered. 2013-02-08 16:36:50,756 INFO org.apache.hadoop.ipc.Server: Starting SocketReader 2013-02-08 16:36:50,758 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort50020 registered. 2013-02-08 16:36:50,758 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort50020 registered. 2013-02-08 16:36:50,760 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration = DatanodeRegistration(ncepspa117.nce.amadeus.net:50010, storageID=, infoPort=50075, ipcPort=50020) Here the logs of the namenode: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = ncepspa119/172.16.140.119 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.1.1 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.1 -r 1411108; compiled by 'hortonfo' on Mon Nov 19 10:48:11 UTC 2012 ************************************************************/ 2013-02-08 16:36:48,124 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2013-02-08 16:36:48,136 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2013-02-08 16:36:48,137 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2013-02-08 16:36:48,137 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system started 2013-02-08 16:36:48,287 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2013-02-08 16:36:48,297 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered. 2013-02-08 16:36:48,298 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source NameNode registered. 2013-02-08 16:36:48,323 INFO org.apache.hadoop.hdfs.util.GSet: VM type = 64-bit 2013-02-08 16:36:48,323 INFO org.apache.hadoop.hdfs.util.GSet: 2% max memory = 17.77875 MB 2013-02-08 16:36:48,323 INFO org.apache.hadoop.hdfs.util.GSet: capacity = 2^21 = 2097152 entries 2013-02-08 16:36:48,323 INFO org.apache.hadoop.hdfs.util.GSet: recommended=2097152, actual=2097152 2013-02-08 16:36:48,359 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=psporacle 2013-02-08 16:36:48,360 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup 2013-02-08 16:36:48,360 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled=true 2013-02-08 16:36:48,366 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.bl |