Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - Re: DFS filesystem used for Standalone and Pseudo-Distributed operation


+
Mohammad Tariq 2013-01-17, 13:58
+
Vikas Jadhav 2013-01-17, 15:09
Copy link to this message
-
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
Mohammad Tariq 2013-01-17, 15:08
Hello Glen,

  That is the default behavior. I is advisable to include "hadoop.tmp.dir"
property in your core-site.xml file and "dfs.name.dir"  &  "dfs.data.dir"
properties in your hdfs-site.xml in order to avoid any problem when you
reboot the machine. Looks like you have already achieved the goal. But,
if you still need any config related help you can visit this
link<http://cloudfront.blogspot.in/2012/07/how-to-configure-hadoop.html#.UPgTlZs-vl8>.
I have tries
to explain the procedure there along with the reasons behind adding the
config properties.

HTH

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <[EMAIL PROTECTED]> wrote:

>  On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
>       Pl find my comments embedded below :
>
>  1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem?  (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
>  >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does?  I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
>  >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp?  It's subject
> that way to getting erased each time I reboot my computer.  Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
>  Warm Regards,
> Tariq
> https://mtariq.jux.com/
>  cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <[EMAIL PROTECTED]> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem?  (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does?  I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>