|
|
+
Mohammad Tariq 2013-01-17, 13:58
-
Re: DFS filesystem used for Standalone and Pseudo-Distributed operationVikas Jadhav 2013-01-17, 15:09
You can control whr to store hdfs data set path of following property in
core-site.xml also u can refer to Yahoo tutorial http://developer.yahoo.com/hadoop/tutorial/module2.html <configuration> <property> <name>fs.default.name</name> <value>hdfs://*your.server.name.com*:9000</value> </property> <property> <name>dfs.data.dir</name> <value>/home/*username*/hdfs/data</value> </property> <property> <name>dfs.name.dir</name> <value>/home/*username*/hdfs/name</value> </property> </configuration> On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <[EMAIL PROTECTED]> wrote: > On 01/17/2013 08:58 AM, Mohammad Tariq wrote: > > Hello Glen, > > Pl find my comments embedded below : > > 1.) The Standalone Operation ( > http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just > to confirm, can run without any DFS filesystem? (We're not being asked to > run "bin/hadoop namenode -format" for it.) > >>Yes. You don't need HDFS for this. > > > 2.) For Pseudo-Distributed Operation ( > http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed), > where we are instructed to run "bin/hadoop namenode -format", question: how > do I undo whatever that command does? I.e., how do I delete that > distributed file system created as well as remove from Hadoop any knowledge > of its prior existence -- do I have to manually delete files with OS > commands (what do I remove?) or is there some type of "bin/hadoop namenode > -delete" command that undoes the "-format" command? > >>You can reformat the NameNode or manually delete the directories > holding the data and the metadata. > > > Thanks, Tariq, looking at the output of the namenode -format command it > looks like the only folder I have to delete is /tmp/hadoop-gmazza: > > 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4, > editlog=/tmp/hadoop-gmazza/dfs/name/current/edits > 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4, > editlog=/tmp/hadoop-gmazza/dfs/name/current/edits > 13/01/17 09:45:08 INFO common.Storage: Storage directory > /tmp/hadoop-gmazza/dfs/name has been successfully formatted. > > Question: Why does Hadoop place the dfs folder under /tmp? It's subject > that way to getting erased each time I reboot my computer. Looking at the > instructions for namenode -format: > http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it > doesn't appear we have an option to specify where we would like the dfs > folder to be created. > > Thanks, > Glen > > > > > > Warm Regards, > Tariq > https://mtariq.jux.com/ > cloudfront.blogspot.com > > > On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <[EMAIL PROTECTED]> wrote: > >> Hi, I'm following the Hadoop 1.1.1 tutorial ( >> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have >> these questions: >> >> 1.) The Standalone Operation ( >> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just >> to confirm, can run without any DFS filesystem? (We're not being asked to >> run "bin/hadoop namenode -format" for it.) >> >> 2.) For Pseudo-Distributed Operation ( >> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed), >> where we are instructed to run "bin/hadoop namenode -format", question: how >> do I undo whatever that command does? I.e., how do I delete that >> distributed file system created as well as remove from Hadoop any knowledge >> of its prior existence -- do I have to manually delete files with OS >> commands (what do I remove?) or is there some type of "bin/hadoop namenode >> -delete" command that undoes the "-format" command? >> >> Thanks, >> Glen >> >> -- >> Glen Mazza >> Talend Community Coders - coders.talend.com >> blog: www.jroller.com/gmazza >> >> > > > -- > Glen Mazza > Talend Community Coders - coders.talend.com > blog: www.jroller.com/gmazza > > -- * * * Thanx and Regards* * Vikas Jadhav* +
Mohammad Tariq 2013-01-17, 15:08
|