Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: DFS filesystem used for Standalone and Pseudo-Distributed operation


Copy link to this message
-
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
You can control whr to store hdfs data set path of following property in
core-site.xml

also u can refer to Yahoo tutorial
http://developer.yahoo.com/hadoop/tutorial/module2.html
<configuration>
  <property>
    <name>fs.default.name</name>

    <value>hdfs://*your.server.name.com*:9000</value>
  </property>
  <property>
    <name>dfs.data.dir</name>

    <value>/home/*username*/hdfs/data</value>
  </property>
  <property>
    <name>dfs.name.dir</name>

    <value>/home/*username*/hdfs/name</value>
  </property>
</configuration>

On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <[EMAIL PROTECTED]> wrote:

>  On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
>       Pl find my comments embedded below :
>
>  1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem?  (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
>  >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does?  I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
>  >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp?  It's subject
> that way to getting erased each time I reboot my computer.  Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
>  Warm Regards,
> Tariq
> https://mtariq.jux.com/
>  cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <[EMAIL PROTECTED]> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem?  (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does?  I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
--
*
*
*

Thanx and Regards*
* Vikas Jadhav*