Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - namenode directory disappear after machines restart


Copy link to this message
-
Re: namenode directory disappear after machines restart
Mohammad Tariq 2012-05-22, 09:48
That's great..The best way to get these kind of info is to ask
questions on the mailing list whenever we face any problem.There are
certain things that are not documented anywhere. I had faced a lot of
problems initially, but the community and the people are really
great.There are so many experienced people that we get the answers for
almost all of our problems.(I am also planning to write a blog this
weekend that addresses these kind of problems so that people like us
do not face the problems that we have already faced).

Regards,
    Mohammad Tariq
On Tue, May 22, 2012 at 3:04 PM, Brendan cheng <[EMAIL PROTECTED]> wrote:
>
> Thanks and it works!
> I wonder where can we find all the settings.  I check the code for hdfs-default.xml but it doesn't have the settings you mentioned.
> Brendan
>
> ----------------------------------------
>> From: [EMAIL PROTECTED]
>> Date: Tue, 22 May 2012 13:03:17 +0530
>> Subject: Re: namenode directory disappear after machines restart
>> To: [EMAIL PROTECTED]
>>
>> Hello Brendan,
>>
>> Do as suggested by Marcos..If you do not set these properties,
>> Hadoop uses "tmp" directory by default..Apart from setting these
>> properties in your hdfs-site.xml file add the following property in
>> your core-site.xml file -
>>
>> <property>
>> <name>hadoop.tmp.dir</name>
>> <value>some_permanent_path</value>
>> </property>
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>> On Tue, May 22, 2012 at 10:54 AM, Marcos Ortiz <[EMAIL PROTECTED]> wrote:
>> > This is an usual behavior on Unix/Linux systems. When you restart the
>> > system, the content of the /tmp directory is cleaned, because precisely, the
>> > purpose of this directory is to keep files temporally.
>> > For that reason, the data directory for the HDFS filesystem should be
>> > another directory, /var/hadoop/data for example, of course, a directory
>> > durable in time.
>> > So, you should change your dfs.name.dir and your dfs.data.dir variable in
>> > your hdfs-site.xml.
>> >
>> > Regards
>> >
>> >
>> > On 05/21/2012 11:21 PM, Brendan cheng wrote:
>> >>
>> >> Hi,
>> >> I'm not sure if there is a setting to avoid the Namenode removed after
>> >> hosting machine of Namenode restart.I found that after successfully
>> >> installed single node pseudo distributed hadoop following from your website,
>> >> the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine
>> >> reboot.
>> >> What do I miss?
>> >> Brendan
>> >> 2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage:
>> >> Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22
>> >> 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> >> FSNamesystem initialization
>> >> failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>> >> Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage
>> >> directory does not exist or is not accessible.    at
>> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>> >>       at
>> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>> >>        at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>> >>    at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>> >>        at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>> >>  at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>> >>   at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22
>> >> 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> >> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
>> >> /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory
>> >> does not exist or is not accessible. at