Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - auto-added RegionServers


Copy link to this message
-
Re: auto-added RegionServers
Bryan Beaudreault 2012-05-24, 05:07
We use puppet and ldap to assign cluster names to servers as we spin them in ec2. Our configs are pushed by fabric, which is organized by these cluster names. This makes it harder to assign a server to the wrong hmaster because it will automatically get the configs for the cluster it is in.

Additionally, we use ec2 security groups to ensure different clusters are sandboxed from each other. You could get this same benefit using VPN or firewall .

Sent from iPhone.

On May 23, 2012, at 11:30 PM, Norbert Burger <[EMAIL PROTECTED]> wrote:

> We had a situation earlier today in our PROD cluster where a test machine
> was accidentally configured with our PROD cluster config.  On startup, the
> HMaster promptly accepted the RS into the fold, and started re-assigning
> regions to it.  The mass migration caused write latencies to increase, and
> it eventually took an HMaster restart to bring things back to normal.
>
> DFS has the dfs.hosts conf setting, which dictates which datanodes are
> allowed to join.  In our setup, we're managing dfs.hosts via configuration
> management.  From what I can tell looking through
> hbase/master/ServerManager.java, there is no equivalent setting on the
> HBase side.  Do folks already rely on this auto-add feature, or would it be
> helpful if there was a similar stop-gap config param for regionservers?
>
> Norbert