Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - hdfs file - datanode association


+
Vincent Boucher 2011-10-06, 09:19
Copy link to this message
-
Re: hdfs file - datanode association
John George 2011-10-06, 13:01
AFAIK, it might not be possible with a single namenode. You could look
into federated namenodes - but I do not think even federated NN would
allow that today (I might be wrong though).

If you have two different NNs, why would you need separate libs etc? You
could point to the respective namenode while accessing that namenode.

-John George

On 10/6/11 4:19 AM, "Vincent Boucher" <[EMAIL PROTECTED]> wrote:

>Hello,
>
>We are wondering if it is possible to require that the namenode redirects
>the
>blocks of the files of a given directory to a particular set of datanodes?
>
>Our case is the following:
>
> - Servers
>    10 x mass storage servers of each 50TB, RAID6
>       -> 500TB available for hdfs
>    30 x working nodes with 6TB (no RAID)
>       -> 180TB available for hdfs
>
>We'd like that the files stored in
>   /hdfs/ms
>be hosted on the mass storage (ms) servers,
>while the files in
>  /hdfs/wn
>be hosted on the working nodes (wn).
>
>Having 2 namenodes, one for each subdir /hdfs/ms or /hdfs/wn would do the
>job.
>However, at the client level it means dealing with to hadoop
>installation/libs ... not great.
>
>It is possible to do it with a single namenode?
>
>Cheers,
>
>Vincent
+
Will Maier 2011-10-06, 13:12
+
Vincent Boucher 2011-10-06, 15:44
+
Vincent Boucher 2011-10-11, 08:08