Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: HDFS to a different location other than HADOOP HOME


+
Raj Hadoop 2013-06-14, 17:40
Copy link to this message
-
Re: HDFS to a different location other than HADOOP HOME
Mohammad Tariq 2013-06-14, 17:44
Change the permissions of /SD1/hadoop_data to 755 and restart the process.

Warm Regards,
Tariq
cloudfront.blogspot.com
On Fri, Jun 14, 2013 at 11:10 PM, Raj Hadoop <[EMAIL PROTECTED]> wrote:

> I have modified dfs.data.dir from the default value to another value which
> is outside 'HADOOP_HOME' and it is >/SD1/hadoop_data.
>
> <property>
> <name>dfs.data.dir</name>
> <value>/SD1/hadoop_data</value>
> <description>Where DataNodes store their blocks</description>
> </property>
>
> My data node is not starting after the above change. Can anyone tell whats
> the issue?
>
>   *From:* Sandeep L <[EMAIL PROTECTED]>
> *To:* "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> *Sent:* Wednesday, June 12, 2013 1:48 AM
> *Subject:* RE: HDFS to a different location other than HADOOP HOME
> **
>  Instead of using *${hadoop.tmp.dir}/dfs/data* you give multiple
> directories with absolute paths.
> Just try as follows:
>  <property>
> <name>hadoop.tmp.dir</name>
> <value>/home1/user1/dir1,/home2/user2/dir2</value>
> </property>
> This will allow HDFS to write data into all specified directories.
> **
>  From: [EMAIL PROTECTED]**Date: Tue, 11 Jun 2013 23:14:08 +0530**Subject:
> Re: HDFS to a different location other than HADOOP HOME**To:
> [EMAIL PROTECTED]
> ****
> Thank you Pramod and Shahab.
> **
> Warm Regards,
> Tariq
> cloudfront.blogspot.com
> ****
> On Tue, Jun 11, 2013 at 11:03 PM, Shahab Yunus <[EMAIL PROTECTED]>wrote:
> **
>
> http://hadoop.apache.org/docs/stable/hdfs-default.html**
>   dfs.data.dir ${hadoop.tmp.dir}/dfs/data Determines where on the local
> filesystem an DFS data node should store its blocks. If this is a
> comma-delimited list of directories, then data will be stored in all named
> directories, typically on different devices. Directories that do not exist
> are ignored.
>  ****
> On Tue, Jun 11, 2013 at 1:08 PM, Raj Hadoop <[EMAIL PROTECTED]> wrote:**
>
>  Hi Tariq,
>
> What is the default value of dfs.data.dir? My hdfs-site.xml doesnt have
> this value defined. So what is the default value?
>
> Thanks,
> Raj
> **
>   *From:* Mohammad Tariq <[EMAIL PROTECTED]>
> *To:* "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; Raj Hadoop <
> [EMAIL PROTECTED]>
> *Sent:* Tuesday, June 11, 2013 12:49 PM
> *Subject:* Re: HDFS to a different location other than HADOOP HOME
> **
> Hello Raj,
> **
>     Although the way which Mustaqeem has told is correct, there is more
> appropriate way to do this. Add the "dfs.data.dir" property in your
> hdfs-site.xml and give the "path you want to use for HDFS data storage" as
> its value. The directory pointed by hadoop.tmp.dir also contains the
> metadata and some other temporary files, so it's always better to have a
> dedicated location to store the actual HDFS data.
>  **
> Warm Regards,
> Tariq
> cloudfront.blogspot.com**
> ****
> On Tue, Jun 11, 2013 at 8:12 PM, Raj Hadoop <[EMAIL PROTECTED]> wrote:**
>
>
> Thanks Mustaqeem.
>
> hadoop.tmp.dir - Can this store HDFS files? I mean - is there any
> difference in files from the one we create under HADOOP_HOME.
>
>
> **
>   *From:* Mohammad Mustaqeem <[EMAIL PROTECTED]>
> *To:* user <[EMAIL PROTECTED]>; Raj Hadoop <[EMAIL PROTECTED]>
> *Sent:* Tuesday, June 11, 2013 10:28 AM
> *Subject:* Re: HDFS to a different location other than HADOOP HOME
>  **
> You can mentioned the "hadoop.tmp.dir" property in the core-site.xml as -
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/app/software/app1</value>
> </property>
> "/app/software/app1" will be used as location for storing the hdfs files.
> **
> ****
> On Tue, Jun 11, 2013 at 7:52 PM, Raj Hadoop <[EMAIL PROTECTED]> wrote:**
>
>  Hi,
>
> I have a one node Hadoop cluster for my POC. The HADOOP_HOME is under a
> directory
>
> /usr/home/hadoop
>
> I dont have much space on /usr and want to use other accounts/applications
> storage which are located at /app/software/app1 etc.,
>
> How can I use the /app/software/app1 location for my HDFS files and
> storage. Is it possible? Please advise.