Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> DFS filesystem used for Standalone and Pseudo-Distributed operation


Copy link to this message
-
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
> Hello Glen,
>
>      Pl find my comments embedded below :
>
> 1.) The Standalone Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
> just to confirm, can run without any DFS filesystem?  (We're not being
> asked to run "bin/hadoop namenode -format" for it.)
> >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation
> (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format",
> question: how do I undo whatever that command does?  I.e., how do I
> delete that distributed file system created as well as remove from
> Hadoop any knowledge of its prior existence -- do I have to manually
> delete files with OS commands (what do I remove?) or is there some
> type of "bin/hadoop namenode -delete" command that undoes the
> "-format" command?
> >>You can reformat the NameNode or manually delete the directories holding
> the data and the metadata.

Thanks, Tariq, looking at the output of the namenode -format command it
looks like the only folder I have to delete is /tmp/hadoop-gmazza:

13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
13/01/17 09:45:08 INFO common.Storage: Storage directory
/tmp/hadoop-gmazza/dfs/name has been successfully formatted.

Question: Why does Hadoop place the dfs folder under /tmp?  It's subject
that way to getting erased each time I reboot my computer. Looking at
the instructions for namenode -format:
http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
doesn't appear we have an option to specify where we would like the dfs
folder to be created.

Thanks,
Glen

>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com <http://cloudfront.blogspot.com>
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     Hi, I'm following the Hadoop 1.1.1 tutorial
>     (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and
>     I have these questions:
>
>     1.) The Standalone Operation
>     (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local),
>     just to confirm, can run without any DFS filesystem?  (We're not
>     being asked to run "bin/hadoop namenode -format" for it.)
>
>     2.) For Pseudo-Distributed Operation
>     (http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>     where we are instructed to run "bin/hadoop namenode -format",
>     question: how do I undo whatever that command does?  I.e., how do
>     I delete that distributed file system created as well as remove
>     from Hadoop any knowledge of its prior existence -- do I have to
>     manually delete files with OS commands (what do I remove?) or is
>     there some type of "bin/hadoop namenode -delete" command that
>     undoes the "-format" command?
>
>     Thanks,
>     Glen
>
>     --
>     Glen Mazza
>     Talend Community Coders - coders.talend.com <http://coders.talend.com>
>     blog: www.jroller.com/gmazza <http://www.jroller.com/gmazza>
>
>
--
Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza