Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - adding space on existing datanode ?


+
brice lecomte 2013-02-22, 13:57
+
Jean-Marc Spaggiari 2013-02-22, 18:14
Copy link to this message
-
Re: adding space on existing datanode ?
Patai Sangbutsarakum 2013-02-22, 18:56
Just want to add up from JM.

If you already have balancer run in cluster every day, that will help the new drive(s) get balanced.

P

From: Jean-Marc Spaggiari <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Fri, 22 Feb 2013 13:14:14 -0500
To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: adding space on existing datanode ?

 add disk space to you datanode you simply need to add another drive, then add it to the dfs.data.dir or dfs.datanode.data.dir entry. After a datanode restart, hadoop will start to use it.

It will not balance the existing data between the directories. It will continue to add to the 2. If one goes full, it will only continue with the other one. If required, you can balance the data manually. Or depending on your use case and the options you have, you can stop the datanode, delete the content of the 2 data directories and restart it. It will stat to receive data to duplicate and will share it evenly between the 2 directories. This last solution is not recommended. But for a test environment it might be easier.
+
Jean-Marc Spaggiari 2013-02-25, 12:15