Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Can we declare some HDFS nodes "primary"


Copy link to this message
-
Can we declare some HDFS nodes "primary"
David Parks 2012-12-11, 11:39
Assume for a moment that you have a large cluster of 500 AWS spot instance
servers running. And you want to keep the bid price low, so at some point
it's likely that the whole cluster will get axed until the spot price comes
down some.

 

In order to maintain HDFS continuity I'd want say 10 servers running as
normal instances, and I'd want to ensure that HDFS is replicating 100% of
data to those 10 that don't run the risk of group elimination.

 

Is it possible for HDFS to ensure replication to these "primary" nodes?