Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> DFS filesystem used for Standalone and Pseudo-Distributed operation


+
Glen Mazza 2013-01-17, 13:54
Copy link to this message
-
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
> Hello Glen,
>
>      Pl find my comments embedded below :
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem?  (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does?  I.e., how do I
> delete that distributed file system created as well as remove from
> Hadoop any knowledge of its prior existence -- do I have to manually
> delete files with OS commands (what do I remove?) or is there some
> type of "bin/hadoop namenode -delete" command that undoes the
> "-format" command?
> >>You can reformat the NameNode or manually delete the directories holding
> the data and the metadata.

Thanks, Tariq, looking at the output of the namenode -format command it
looks like the only folder I have to delete is /tmp/hadoop-gmazza:

13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO common.Storage: Storage directory
/tmp/hadoop-gmazza/dfs/name has been successfully formatted.

Question: Why does Hadoop place the dfs folder under /tmp?  It's subject
that way to getting erased each time I reboot my computer. Looking at
the instructions for namenode -format:
http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
doesn't appear we have an option to specify where we would like the dfs
folder to be created.

Thanks,
Glen

>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com <http://cloudfront.blogspot.com>
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     Hi, I'm following the Hadoop 1.1.1 tutorial
>     (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and
>     I have these questions:
>
>     1.) The Standalone Operation
>     (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
>     just to confirm, can run without any DFS filesystem?  (We're not
>     being asked to run "bin/hadoop namenode -format" for it.)
>
>     2.) For Pseudo-Distributed Operation
>     (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>     where we are instructed to run "bin/hadoop namenode -format",
>     question: how do I undo whatever that command does?  I.e., how do
>     I delete that distributed file system created as well as remove
>     from Hadoop any knowledge of its prior existence -- do I have to
>     manually delete files with OS commands (what do I remove?) or is
>     there some type of "bin/hadoop namenode -delete" command that
>     undoes the "-format" command?
>
>     Thanks,
>     Glen
>
>     --
>     Glen Mazza
>     Talend Community Coders - coders.talend.com <http://coders.talend.com>
>     blog: www.jroller.com/gmazza <http://www.jroller.com/gmazza>
>
>
--
Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB