Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: DFS filesystem used for Standalone and Pseudo-Distributed operation


Copy link to this message
-
Re: DFS filesystem used for Standalone and Pseudo-Distributed operation
You can control whr to store hdfs data set path of following property in
core-site.xml

also u can refer to Yahoo tutorial
http://developer.yahoo.com/hadoop/tutorial/module2.html
<configuration>
  <property>
    <name>fs.default.name</name>

    <value>hdfs://*your.server.name.com*:9000</value>
  </property>
  <property>
    <name>dfs.data.dir</name>

    <value>/home/*username*/hdfs/data</value>
  </property>
  <property>
    <name>dfs.name.dir</name>

    <value>/home/*username*/hdfs/name</value>
  </property>
</configuration>

On Thu, Jan 17, 2013 at 8:31 PM, Glen Mazza <[EMAIL PROTECTED]> wrote:

>  On 01/17/2013 08:58 AM, Mohammad Tariq wrote:
>
> Hello Glen,
>
>       Pl find my comments embedded below :
>
>  1.) The Standalone Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
> to confirm, can run without any DFS filesystem?  (We're not being asked to
> run "bin/hadoop namenode -format" for it.)
>  >>Yes. You don't need HDFS for this.
>
>
> 2.) For Pseudo-Distributed Operation (
> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
> where we are instructed to run "bin/hadoop namenode -format", question: how
> do I undo whatever that command does?  I.e., how do I delete that
> distributed file system created as well as remove from Hadoop any knowledge
> of its prior existence -- do I have to manually delete files with OS
> commands (what do I remove?) or is there some type of "bin/hadoop namenode
> -delete" command that undoes the "-format" command?
>  >>You can reformat the NameNode or manually delete the directories
> holding the data and the metadata.
>
>
> Thanks, Tariq, looking at the output of the namenode -format command it
> looks like the only folder I have to delete is /tmp/hadoop-gmazza:
>
> 13/01/17 09:45:08 INFO namenode.FSEditLog: closing edit log: position=4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO namenode.FSEditLog: close success: truncate to 4,
> editlog=/tmp/hadoop-gmazza/dfs/name/current/edits
> 13/01/17 09:45:08 INFO common.Storage: Storage directory
> /tmp/hadoop-gmazza/dfs/name has been successfully formatted.
>
> Question: Why does Hadoop place the dfs folder under /tmp?  It's subject
> that way to getting erased each time I reboot my computer.  Looking at the
> instructions for namenode -format:
> http://hadoop.apache.org/docs/r1.1.0/commands_manual.html#namenode, it
> doesn't appear we have an option to specify where we would like the dfs
> folder to be created.
>
> Thanks,
> Glen
>
>
>
>
>
>  Warm Regards,
> Tariq
> https://mtariq.jux.com/
>  cloudfront.blogspot.com
>
>
> On Thu, Jan 17, 2013 at 7:24 PM, Glen Mazza <[EMAIL PROTECTED]> wrote:
>
>> Hi, I'm following the Hadoop 1.1.1 tutorial (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html) and I have
>> these questions:
>>
>> 1.) The Standalone Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#Local), just
>> to confirm, can run without any DFS filesystem?  (We're not being asked to
>> run "bin/hadoop namenode -format" for it.)
>>
>> 2.) For Pseudo-Distributed Operation (
>> http://hadoop.apache.org/docs/r1.1.1/single_node_setup.html#PseudoDistributed),
>> where we are instructed to run "bin/hadoop namenode -format", question: how
>> do I undo whatever that command does?  I.e., how do I delete that
>> distributed file system created as well as remove from Hadoop any knowledge
>> of its prior existence -- do I have to manually delete files with OS
>> commands (what do I remove?) or is there some type of "bin/hadoop namenode
>> -delete" command that undoes the "-format" command?
>>
>> Thanks,
>> Glen
>>
>> --
>> Glen Mazza
>> Talend Community Coders - coders.talend.com
>> blog: www.jroller.com/gmazza
>>
>>
>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
--
*
*
*

Thanx and Regards*
* Vikas Jadhav*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB