Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Can we declare some HDFS nodes "primary"


Copy link to this message
-
Can we declare some HDFS nodes "primary"
Assume for a moment that you have a large cluster of 500 AWS spot instance
servers running. And you want to keep the bid price low, so at some point
it's likely that the whole cluster will get axed until the spot price comes
down some.

 

In order to maintain HDFS continuity I'd want say 10 servers running as
normal instances, and I'd want to ensure that HDFS is replicating 100% of
data to those 10 that don't run the risk of group elimination.

 

Is it possible for HDFS to ensure replication to these "primary" nodes?

 

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB