Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: datanode has no storageID


Copy link to this message
-
Re: datanode has no storageID
> 2. how/where does the namenode stores the datanodes's storageids ?

When the datanode connects with the namenode for the first time it will register with the namenode and during the registration the of the datanode the storage id for the datanode is generated.

>
> 4. can I format/reset the namenode to enable the datanode to reconnect ?

If you do not care about the existing data you can certainly do that. Make sure you stop all processes, format the namenode, clean up the data directories on all your datanodes (dfs.data.dir prop in hdfs-site.xml) and then start the namenode and the datanodes.

--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/

On Aug 28, 2012, at 9:13 AM, [EMAIL PROTECTED] wrote:

> Hi,
> hope it's not a newby question...
> I installed several versions of hadoop for testing,
> (0.20.203, 0.21.0, and 1.0.3)
> on various machines.
> now I am using 1.0.3 on all the machines,
> I face a problem that in some of the machhines, the datanode gets no
> storageID from the namenode.
> where it works, the datanode has the following lines in the log file:
> (and current/VERSION has a storageID=< some ID > )
> ---------------------------------------------------------------
> 2012-08-28 19:04:31,415 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration = DatanodeRegistration(datanode-works.cs.tau.ac.il:50010, storageID=DS-996163017-<machines-ip>-50010-1342683478942, infoPort=50075, ipcPort=50020)
> 2012-08-28 19:04:31,418 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting asynchronous block report scan
> 2012-08-28 19:04:31,418 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(<machines-ip>:50010, storageID=DS-996163017-<machines-ip>-50010-1342683478942, infoPort=50075, ipcPort=50020)In DataNode.run, data = FSDataset{dirpath='/var/cache/hdfs/hadoop-data-node/current'}
> 2012-08-28 19:04:31,419 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> -------------------------------------------------------------------------------
>
> where it doesn't work I have only the first line and it hangs.
> (and current/VERSION has a 'storageID=' empty value line )
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 2012-08-28 18:42:01,297 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration = DatanodeRegistration(machinename.cs.tau.ac.il:50010, storageID=, infoPort=50075, ipcPort=50020)
> 2012-08-28 18:42:01,287 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1. any Ideas?
> 2. how/where does the namenode stores the datanodes's storageids ?
> 3. how can I get a new storageid for a datanode or it's old ID ?
> 4. can I format/reset the namenode to enable the datanode to reconnect ?
>
> thanks!
> -
> Boaz Yarom
> CS System Team
> 03-640-8961 / 7637
>
>

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB