Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Namenode in inconsistent state: how to reinitialize the storage directory


Copy link to this message
-
Re: Namenode in inconsistent state: how to reinitialize the storage directory
Brock Noland 2011-10-26, 10:48
Moving this discussion to CDH-USER since it sounds like it's the Cloudera VM.

BCC mapreduce-user

On Wed, Oct 26, 2011 at 2:17 AM, Stephen Boesch <[EMAIL PROTECTED]> wrote:
> I found a suggestion to reformat the namenode.  In order to do so, I found
> it necessary to set the dir to 777. AFter
>
> $ sudo chmod 777 /var/lib/hadoop-0.20/cache/hadoop/dfs/name
> $ ./hadoop namenode -format

What user was this executed as?  In the VM & CDH package install
NameNode/DataNode run as hdfs while JobTracker/TaskTracker run as
mapred.

As such, if you executed the format command as:

sudo -u hdfs

You shouldn't need 777 permissions.

> (successful)
> $ ./hadoop-daemon.sh --config $HADOOP/conf start namenode
> (success!)

CDH comes with init startup scripts:

/etc/init.d/hadoop-0.20-namenode
/etc/init.d/hadoop-0.20-datanode
/etc/init.d/hadoop-0.20-jobtracker
/etc/init.d/hadoop-0.20-tasktracker
/etc/init.d/hadoop-0.20-secondarynamenode

However, based on the commands you have executed above, at this point,
I think you'd need to some chow'ing to use the init scripts.

> So.. this leads to a related question:  What gives with these permissions?
>    Maybe  this is cloudera specific.   I am logged in to cloudera user,. but
> these directories have owners/groups with a mix of hadoop, mapred, hbase,
> hdfs, etc.    When i look in /etc/passwd and /etc/group there is no clear
> indication that cloudera should be able to access files owned by members of
> those groups.
> Where is there more info about making the file permissions happy when
> running the various hadoop services from cloudera user ?
> i am on CDH3u1

NameNode/DataNode run as hdfs while JobTracker/TaskTracker run as mapred

Brock