Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Misconfiguration of hdfs-site.xml


+
Jean-Marc Spaggiari 2012-12-18, 20:28
+
Harsh J 2012-12-18, 20:36
+
Jean-Marc Spaggiari 2012-12-18, 20:53
Copy link to this message
-
Re: Misconfiguration of hdfs-site.xml
Bryan Beaudreault 2012-12-18, 20:36
Updating the config and restarting will result in data loss.  You're going
to want to migrate the data from your tmp directory to the appropriate
place.

The NameNode will use the dfs.name.dir property, and DataNodes will use the
dfs.data.dir.  Assuming you aren't running a DN and NN on the same server
(don't do that), then they shouldn't collide/overlap.  Even still its
probably best to use different directories.  For instance mine is like this:

dfs.name.dir = /mnt/hdfs/name,/mnt2/hdfs/name
dfs.data.dir
= /mnt/hdfs/data,/mnt2/hdfs/data,/mnt3/hdfs/data,/mnt4/hdfs/data

Hope this helps.
On Tue, Dec 18, 2012 at 3:28 PM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> For months now I'm using my hadoop cluster with absolutly nothing
> related to the drive directory on my hdfs-site.xml file.
>
> It seems that it's using the hadoop.tmp.dir directory to store data.
>
> My hadoop.tmp.dir is pointing to
> /home/hadoop/haddop_drive/${user.name} and on my
> /home/hadoop/haddop_drive directory I can see hadoop and hbase.
>
> Now, I want to configure that properly without loosing the data I have.
>
> I'm using 1.0.3.
>
> Based on the documentation
> http://hadoop.apache.org/docs/r1.0.3/cluster_setup.html#Site+Configuration
> it seems I need to setup dfs.data.dir to point to
> /home/hadoop/haddop_drive. If I add this entry and restart my
> datanode, will it "simply" continue to point to the same place and
> find the data he needs?
>
> Also, what about dfs.name.dir ? I guess I can simply point it to the
> same place, right?
>
> On  /home/hadoop/haddop_drive/hadoop/dfs I have data and name
> directories. So I'm not sure if I should point the properties above to
>  /home/hadoop/haddop_drive or to  /home/hadoop/haddop_drive/dfs.
>
> Thanks,
>
> JM
>