Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: HDFS Installation / Configuration


Copy link to this message
-
Re: HDFS Installation / Configuration
Hi Yuvraj,

      Once the disk is mounted we just need to give the names of all
the directories as a comma separated value for 'dfs.data.dir'
property, and the data will be sent to all these places. And we don't
have to worry as far as writing the data to a new storage is
concerned. As soon as additional resources are added to the cluster
the metadata is updated. Namenode takes care of all that.

Regards,
    Mohammad Tariq
On Wed, Jul 18, 2012 at 1:33 PM, Yuvrajsinh Chauhan
<[EMAIL PROTECTED]> wrote:
> Dear Tariq,
>
> My current setup is as below:
>
> Node-1  Node-2
> /       40GB    /       40GB
> /opt    120GB   /opt    120GB
> /u01    95GB    /u01    95GB
> /dev/shm        19GB    /dev/shm        19GB
> Common Storage
>         /DATA1  100GB
>         /DATA2  100GB
>         /DATA3  100GB
>
> As per my current configurations, Data node pointed to /usr/local directory.
> And the same was located on Local HDD.
> Now, I want to add /DATA1 partition.
>
> Please let me know which steps I have to follow.
>
> Also, Please let me know, How my application write file directly on Hadoop
> partition, which available on second node too.
>
>
> Regards,
>
> Yuvrajsinh Chauhan || Sr. DBA || CRESTEL-PSG
> Elitecore Technologies Pvt. Ltd.
> 904, Silicon Tower || Off C.G.Road
> Behind Pariseema Building || Ahmedabad || INDIA
> [GSM]: +91 9727746022
>
> -----Original Message-----
> From: Yuvrajsinh Chauhan [mailto:[EMAIL PROTECTED]]
> Sent: 17 July 2012 18:56
> To: '[EMAIL PROTECTED]'
> Subject: RE: HDFS Installation / Configuration
>
> Dear Tariq / Bijoy,
>
> Please Ignore my previous mail. I can now see the directories using
> following commands:
>
> [hadoop@rac1 bin]$ ./hadoop fs -ls /
> Found 5 items
> drwxr-xr-x   - hadoop hadoop              0 2012-07-16 14:23 /test
> drwxr-xr-x   - hadoop hadoop              0 2012-07-17 13:29 /test1
> drwxr-xr-x   - hadoop supergroup          0 2012-07-17 18:03 /user
> drwxr-xr-x   - hadoop supergroup          0 2012-07-17 14:11 /usr
> drwxr-xr-x   - hadoop supergroup          0 2012-07-17 17:41 /yuvi
>
> I can see this data from both the nodes.
>
> Now, I am exploring more using by reading help.
>
> Regards,
>
> Yuvrajsinh Chauhan || Sr. DBA || CRESTEL-PSG Elitecore Technologies Pvt.
> Ltd.
> 904, Silicon Tower || Off C.G.Road
> Behind Pariseema Building || Ahmedabad || INDIA
> [GSM]: +91 9727746022
>
>
> -----Original Message-----
> From: Yuvrajsinh Chauhan [mailto:[EMAIL PROTECTED]]
> Sent: 17 July 2012 18:49
> To: [EMAIL PROTECTED]
> Subject: RE: HDFS Installation / Configuration
>
> Dear Tariq,
>
> Values of both the properties are already configured in hdfs-site.xml file.
>
> <property>
> <name>dfs.name.dir</name>
> <value>/usr/local/hadoopstorage/namenode</value>        #Directories created
> with proper Hadoop user permission with R/W.
> <final>true</final>
> </property>
> <property>
> <name>dfs.data.dir</name>
> <value>/usr/local/hadoopstorage/datanode</value>
> <final>true</final>
> </property>
> ===========================================================================> Command for creating a Directory
>
> [hadoop@rac1 bin]$ ./hadoop fs -mkdir /yuvi
> [hadoop@rac1 bin]$
> [hadoop@rac1 bin]$ ./hadoop fs -ls
> ls: Cannot access .: No such file or directory.
> [hadoop@rac1 bin]$
>
> But when I go on to that path, I cannot see any directory. However, I can
> see the directories from GUI.
> Also I cannot find any error in Logs from both the nodes.
>
>
>
> Regards,
>
> Yuvrajsinh Chauhan || Sr. DBA || CRESTEL-PSG Elitecore Technologies Pvt.
> Ltd.
> 904, Silicon Tower || Off C.G.Road
> Behind Pariseema Building || Ahmedabad || INDIA
> [GSM]: +91 9727746022
>
>
> -----Original Message-----
> From: Mohammad Tariq [mailto:[EMAIL PROTECTED]]
> Sent: 17 July 2012 18:22
> To: [EMAIL PROTECTED]
> Subject: Re: HDFS Installation / Configuration
>
> Hi Yuvrajsinh,
>
>         There is absolutely nothing to be sorry for. Have you added thee