Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Can we declare some HDFS nodes "primary"


+
David Parks 2012-12-11, 11:39
+
Colin McCabe 2012-12-11, 23:53
Copy link to this message
-
Re: Can we declare some HDFS nodes "primary"
Rack awareness with replication factor of 3 on files will help.

You could declare two racks, one carrying these 10 nodes, and default rack
for the rest of them, and the rack-aware default block placement policy
will take care of the rest.
On Dec 11, 2012 5:10 PM, "David Parks" <[EMAIL PROTECTED]> wrote:

> Assume for a moment that you have a large cluster of 500 AWS *spot
> instance* servers running. And you want to keep the bid price low, so at
> some point it’s likely that the whole cluster will get axed until the spot
> price comes down some.****
>
> ** **
>
> In order to maintain HDFS continuity I’d want say 10 servers running as
> normal instances, and I’d want to ensure that HDFS is replicating 100% of
> data to those 10 that don’t run the risk of group elimination.****
>
> ** **
>
> Is it possible for HDFS to ensure replication to these “primary” nodes?***
> *
>
> ** **
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB