Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> namenode directory disappear after machines restart


Copy link to this message
-
Re: namenode directory disappear after machines restart
Hello Brendan,

  Do as suggested by Marcos..If you do not set these properties,
Hadoop uses "tmp" directory by default..Apart from setting these
properties in your hdfs-site.xml file add the following property in
your core-site.xml file -

<property>
 <name>hadoop.tmp.dir</name>
 <value>some_permanent_path</value>
</property>

Regards,
    Mohammad Tariq
On Tue, May 22, 2012 at 10:54 AM, Marcos Ortiz <[EMAIL PROTECTED]> wrote:
> This is an usual behavior on Unix/Linux systems. When you restart the
> system, the content of the /tmp directory is cleaned, because precisely, the
> purpose of this directory is to keep files temporally.
> For that reason, the data directory for the HDFS filesystem should be
> another directory, /var/hadoop/data for example, of course, a directory
> durable in time.
> So, you should change your dfs.name.dir and your dfs.data.dir variable in
> your hdfs-site.xml.
>
> Regards
>
>
> On 05/21/2012 11:21 PM, Brendan cheng wrote:
>>
>> Hi,
>> I'm not sure if there is a setting to avoid the Namenode removed after
>> hosting machine of Namenode restart.I found that after successfully
>> installed single node pseudo distributed hadoop following from your website,
>> the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine
>> reboot.
>> What do I miss?
>> Brendan
>> 2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage:
>> Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22
>> 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> FSNamesystem initialization
>> failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>> Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage
>> directory does not exist or is not accessible.    at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>>       at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>>        at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>>    at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>>        at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>>  at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22
>> 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
>> /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory
>> does not exist or is not accessible. at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>>       at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>>        at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>>    at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>>        at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>>  at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
>>
>> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
>> INFORMATICAS...
>> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>>
>> http://www.uci.cu
>> http://www.facebook.com/universidad.uci
>> http://www.flickr.com/photos/universidad_uci
>
>
> --
> Marcos Luis Ortíz Valmaseda
>  Data Engineer&&  Sr. System Administrator at UCI
>  http://marcosluis2186.posterous.com
>  http://www.linkedin.com/in/marcosluis2186
>  Twitter: @marcosluis2186
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB