Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Namenode in inconsistent state: how to reinitialize the storage directory


Copy link to this message
-
Re: Namenode in inconsistent state: how to reinitialize the storage directory
Moving this discussion to CDH-USER since it sounds like it's the Cloudera VM.

BCC mapreduce-user

On Wed, Oct 26, 2011 at 2:17 AM, Stephen Boesch <[EMAIL PROTECTED]> wrote:
> I found a suggestion to reformat the namenode.  In order to do so, I found
> it necessary to set the dir to 777. AFter
>
> $ sudo chmod 777 /var/lib/hadoop-0.20/cache/hadoop/dfs/name
> $ ./hadoop namenode -format

What user was this executed as?  In the VM & CDH package install
NameNode/DataNode run as hdfs while JobTracker/TaskTracker run as
mapred.

As such, if you executed the format command as:

sudo -u hdfs

You shouldn't need 777 permissions.

> (successful)
> $ ./hadoop-daemon.sh --config $HADOOP/conf start namenode
> (success!)

CDH comes with init startup scripts:

/etc/init.d/hadoop-0.20-namenode
/etc/init.d/hadoop-0.20-datanode
/etc/init.d/hadoop-0.20-jobtracker
/etc/init.d/hadoop-0.20-tasktracker
/etc/init.d/hadoop-0.20-secondarynamenode

However, based on the commands you have executed above, at this point,
I think you'd need to some chow'ing to use the init scripts.

> So.. this leads to a related question:  What gives with these permissions?
>    Maybe  this is cloudera specific.   I am logged in to cloudera user,. but
> these directories have owners/groups with a mix of hadoop, mapred, hbase,
> hdfs, etc.    When i look in /etc/passwd and /etc/group there is no clear
> indication that cloudera should be able to access files owned by members of
> those groups.
> Where is there more info about making the file permissions happy when
> running the various hadoop services from cloudera user ?
> i am on CDH3u1

NameNode/DataNode run as hdfs while JobTracker/TaskTracker run as mapred

Brock
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB