Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - /accumulo dir remains after reformatting name node and deleting datanodes


Copy link to this message
-
Re: /accumulo dir remains after reformatting name node and deleting datanodes
Sean Busbey 2014-01-26, 23:44
HDFS will definitely not function without the NameNode. I'm not sure how
the NameNode process will respond if you delete its storage directory out
from under it (that is, while it is running).

What's the output of `alternatives --display hadoop-conf`?

Is txoig-hdfs01 the name of your namenode server?

How are you shutting down HDFS before deleting things?
On Sun, Jan 26, 2014 at 5:10 PM, Kesten Broughton <[EMAIL PROTECTED]>wrote:

> Apparently, there’s something i really don’t understand about hdfs data
> files.
> I assumed that without the name node, hdfs would cease to function, but
> hadoop fs -ls / still shows stuff when i remove the name node.
>
> [root@txoig-hdfs01:~]$ rm -Rf /hadoop/hdfs/namenode
> [root@txoig-hdfs01:~]$ su hdfs
> [hdfs@txoig-hdfs01:root]$ hadoop fs -ls /
> Found 5 items
> drwxr-xr-x   - hdfs   hdfs          0 2014-01-26 13:36 /accumulo
> drwxr-xr-x   - hdfs   hdfs          0 2013-12-19 18:17 /apps
> drwx------   - mapred hdfs          0 2014-01-26 16:39 /mapred
> drwxrwxrwx   - hdfs   hdfs          0 2014-01-26 14:56 /tmp
> drwxr-xr-x   - hdfs   hdfs          0 2013-12-30 15:11 /user
> [hdfs@txoig-hdfs01:root]$ ls /hadoop/hdfs/
> total 8.0K
> drwxr-xr-x. 2 root 4.0K Jan 26 17:02 .
> drwxr-xr-x. 5 root 4.0K Dec 19 18:13 ..
>
>
> [root@txoig-hdfs02:/]$ rm -Rf /hadoop/hdfs/namesecondary
> [root@txoig-hdfs02:/]$ ls /hadoop/hdfs/
> total 8.0K
> drwxr-xr-x. 2 root 4.0K Jan 26 17:04 .
> drwxr-xr-x. 6 root 4.0K Dec 19 18:14 ..
> [root@txoig-hdfs02:/]$ hadoop fs -ls /
> Found 5 items
> drwxr-xr-x   - hdfs   hdfs          0 2014-01-26 13:36 /accumulo
> drwxr-xr-x   - hdfs   hdfs          0 2013-12-19 18:17 /apps
> drwx------   - mapred hdfs          0 2014-01-26 16:39 /mapred
> drwxrwxrwx   - hdfs   hdfs          0 2014-01-26 14:56 /tmp
> drwxr-xr-x   - hdfs   hdfs          0 2013-12-30 15:11 /user
>
> [root@txoig-hdfs03:/]$ ls /hadoop/hdfs/
> total 8.0K
> drwxr-xr-x. 2 root 4.0K Jan 26 16:23 .
> drwxr-xr-x. 6 root 4.0K Dec 19 18:14 ..
>
> cat /etc/hadoop/conf.empty/hdfs-site.xml contains
>
>
>     <property>
>     <name>dfs.name.dir</name>
>     <value>/hadoop/hdfs/namenode</value>
>   </property>
>
>
> [root@txoig-hdfs03:/]$ ls /etc/hadoop
> total 12K
> drwxr-xr-x.  3 root 4.0K Dec 19 18:10 .
> drwxr-xr-x. 98 root 4.0K Jan 26 16:04 ..
> lrwxrwxrwx.  1 hdfs   29 Dec 19 18:10 conf -> /etc/alternatives/hadoop-conf
> drwxr-xr-x.  2 root 4.0K Jan 26 16:55 conf.empty
>
>
>
>
> On 1/26/14, 4:37 PM, "Kesten Broughton" <[EMAIL PROTECTED]> wrote:
>
> >It¹s possible something was running.  I only did an accumulo/stop-all.sh
> >and sometimes i¹ve noticed i have to go to each node to stop-here.sh.
> >
> >[root@txoig-hdfs02:accumulo-1.5.0]$ ls /hadoop/hdfs
> >drwxr-xr-x. 5 hdfs 4.0K Jan 26 16:10 namesecondary
> >
> >
> >[root@txoig-hdfs01:~]$ ls /hadoop/hdfs/
> >drwxr-xr-x. 5 hdfs 4.0K Jan 26 16:10 namenode
> >
> >
> >[root@txoig-hdfs03:~]$ ls /hadoop/hdfs
> >drwxr-xr-x. 4 root 4.0K Jan 26 16:13 namenode
> ># this is weird. Wasn¹t expecting a second name node.  We aren¹t doing HA.
> >
> >
> >But shutting down all accumulo and hdfs and then doing the same procedure,
> >i still see
> >
> >[root@txoig-hdfs01:~]$ hadoop fs -ls /
> >Found 5 items
> >drwxr-xr-x   - hdfs   hdfs          0 2014-01-26 13:36 /accumulo
> >drwxr-xr-x   - hdfs   hdfs          0 2013-12-19 18:17 /apps
> >drwx------   - mapred hdfs          0 2014-01-26 14:32 /mapred
> >drwxrwxrwx   - hdfs   hdfs          0 2014-01-26 14:56 /tmp
> >drwxr-xr-x   - hdfs   hdfs          0 2013-12-30 15:11 /user
> >
> >
> >
> >On 1/26/14, 4:16 PM, "Christopher" <[EMAIL PROTECTED]> wrote:
> >
> >>Was any part of Hadoop or Accumulo still running when you did this?
> >>Are you sure you deleted the right HDFS data directory referenced in
> >>your Hadoop configuration?
> >>
> >>--
> >>Christopher L Tubbs II
> >>http://gravatar.com/ctubbsii
> >>
> >>
> >>On Sun, Jan 26, 2014 at 4:45 PM, Kesten Broughton <[EMAIL PROTECTED]>
> >>wrote:
> >>>>
> >>>>We did experiments using the root dir for both name nodes and