Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> namenode directory disappear after machines restart


+
Brendan cheng 2012-05-22, 03:21
+
Marcos Ortiz 2012-05-22, 05:24
+
Mohammad Tariq 2012-05-22, 07:33
+
Brendan cheng 2012-05-22, 09:34
Copy link to this message
-
Re: namenode directory disappear after machines restart
Brendan,

The hdfs-default.xml does have dfs.name.dir listed:
http://hadoop.apache.org/common/docs/current/hdfs-default.html. The
configuration is also mentioned on the official tutorial:
http://hadoop.apache.org/common/docs/current/cluster_setup.html#Configuration+Files

On Tue, May 22, 2012 at 3:04 PM, Brendan cheng <[EMAIL PROTECTED]> wrote:
>
> Thanks and it works!
> I wonder where can we find all the settings.  I check the code for hdfs-default.xml but it doesn't have the settings you mentioned.
> Brendan
>
> ----------------------------------------
>> From: [EMAIL PROTECTED]
>> Date: Tue, 22 May 2012 13:03:17 +0530
>> Subject: Re: namenode directory disappear after machines restart
>> To: [EMAIL PROTECTED]
>>
>> Hello Brendan,
>>
>> Do as suggested by Marcos..If you do not set these properties,
>> Hadoop uses "tmp" directory by default..Apart from setting these
>> properties in your hdfs-site.xml file add the following property in
>> your core-site.xml file -
>>
>> <property>
>> <name>hadoop.tmp.dir</name>
>> <value>some_permanent_path</value>
>> </property>
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>> On Tue, May 22, 2012 at 10:54 AM, Marcos Ortiz <[EMAIL PROTECTED]> wrote:
>> > This is an usual behavior on Unix/Linux systems. When you restart the
>> > system, the content of the /tmp directory is cleaned, because precisely, the
>> > purpose of this directory is to keep files temporally.
>> > For that reason, the data directory for the HDFS filesystem should be
>> > another directory, /var/hadoop/data for example, of course, a directory
>> > durable in time.
>> > So, you should change your dfs.name.dir and your dfs.data.dir variable in
>> > your hdfs-site.xml.
>> >
>> > Regards
>> >
>> >
>> > On 05/21/2012 11:21 PM, Brendan cheng wrote:
>> >>
>> >> Hi,
>> >> I'm not sure if there is a setting to avoid the Namenode removed after
>> >> hosting machine of Namenode restart.I found that after successfully
>> >> installed single node pseudo distributed hadoop following from your website,
>> >> the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine
>> >> reboot.
>> >> What do I miss?
>> >> Brendan
>> >> 2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage:
>> >> Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22
>> >> 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> >> FSNamesystem initialization
>> >> failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>> >> Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage
>> >> directory does not exist or is not accessible.    at
>> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>> >>       at
>> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>> >>        at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>> >>    at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>> >>        at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>> >>  at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>> >>   at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22
>> >> 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> >> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
>> >> /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory
>> >> does not exist or is not accessible. at
>> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>> >>       at
>> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)

Harsh J
+
Mohammad Tariq 2012-05-22, 09:48
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB