Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - namenode directory disappear after machines restart


Copy link to this message
-
Re: namenode directory disappear after machines restart
Mohammad Tariq 2012-05-22, 07:33
Hello Brendan,

  Do as suggested by Marcos..If you do not set these properties,
Hadoop uses "tmp" directory by default..Apart from setting these
properties in your hdfs-site.xml file add the following property in
your core-site.xml file -

<property>
 <name>hadoop.tmp.dir</name>
 <value>some_permanent_path</value>
</property>

Regards,
    Mohammad Tariq
On Tue, May 22, 2012 at 10:54 AM, Marcos Ortiz <[EMAIL PROTECTED]> wrote:
> This is an usual behavior on Unix/Linux systems. When you restart the
> system, the content of the /tmp directory is cleaned, because precisely, the
> purpose of this directory is to keep files temporally.
> For that reason, the data directory for the HDFS filesystem should be
> another directory, /var/hadoop/data for example, of course, a directory
> durable in time.
> So, you should change your dfs.name.dir and your dfs.data.dir variable in
> your hdfs-site.xml.
>
> Regards
>
>
> On 05/21/2012 11:21 PM, Brendan cheng wrote:
>>
>> Hi,
>> I'm not sure if there is a setting to avoid the Namenode removed after
>> hosting machine of Namenode restart.I found that after successfully
>> installed single node pseudo distributed hadoop following from your website,
>> the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine
>> reboot.
>> What do I miss?
>> Brendan
>> 2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage:
>> Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22
>> 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> FSNamesystem initialization
>> failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>> Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage
>> directory does not exist or is not accessible.    at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>>       at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>>        at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>>    at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>>        at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>>  at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22
>> 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
>> /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory
>> does not exist or is not accessible. at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>>       at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>>        at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>>    at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>>        at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>>  at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>>   at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
>>
>> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
>> INFORMATICAS...
>> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>>
>> http://www.uci.cu
>> http://www.facebook.com/universidad.uci
>> http://www.flickr.com/photos/universidad_uci
>
>
> --
> Marcos Luis Ortíz Valmaseda
>  Data Engineer&&  Sr. System Administrator at UCI
>  http://marcosluis2186.posterous.com
>  http://www.linkedin.com/in/marcosluis2186
>  Twitter: @marcosluis2186