Otis Gospodnetic 2013-05-06, 17:37
Amandeep Khurana 2013-05-06, 17:41
Otis Gospodnetic 2013-05-06, 17:58
Amandeep Khurana 2013-05-06, 18:00
Otis Gospodnetic 2013-05-06, 18:00
Amandeep Khurana 2013-05-06, 18:06
Mike Spreitzer 2013-05-06, 19:18
Otis Gospodnetic 2013-05-06, 20:13
Otis Gospodnetic 2013-05-06, 18:47
Steph Gosling 2013-05-06, 22:26
-Re: HBase cluster over multiple EC2 Availability Zones?
Andrew Purtell 2013-05-07, 00:01
> We do simple 'rack' awareness based on the AZ returned by the metadata
Just want to add a datapoint that I do this too -- for simulating racks
when running test clusters up on EC2 (it corresponds to physical properties
and makes it easy to group instances for nuking a whole "rack") -- and have
encountered no major problems. I can also echo the comments of others that
cross-AZ deployments can see significant delay and variance on the network.
On Mon, May 6, 2013 at 3:26 PM, Steph Gosling <[EMAIL PROTECTED]> wrote:
> On Mon, 6 May 2013 14:47:49 -0400
> Otis Gospodnetic <[EMAIL PROTECTED]> wrote:
> > Yup. I'm suddenly turned off by that penny per GB! :)
> > Ignoring the dollars, it sounds like one would just have to be OK with
> > increased latencies, but technically nothing would break. Doodling
> > our architecture on paper here, I think we may as well just have
> > complete, independent setups in multiple Regions then - I suspect
> > those pennies add up faster than one would think.
> > Otis
> I run a couple of small HBase clusters (low double-digit nodes each),
> and both span AZs in their respective regions (we're not doing any
> inter-region stuff yet, nor do I expect to,TBH). What AWS don't tell
> you is that not all instance types are available in all AZs,
> particularly for the newer or more esoteric instances.
> We care about this data so any performace hit (not that we've
> particularly noticed one) because of cross AZ traffic is acceptable. We
> do simple 'rack' awareness based on the AZ returned by the metadata
> server, you could probably fine-tune that based on subnet if your
> cluster got big but we've not had the need to.
> With regards to performance specifically, I've not looked explicitly
> but I'd expect that you'll see far more variance based on things like
> instance size, your neighbours on the same host and their behaviour.
> Finally, I'm also surprised about the inter-AZ data charges, that seems
> to be a very wide-spread misconception, and yeah I'd imagine the
> pennies do add up...
> Steph Gosling <[EMAIL PROTECTED]>
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Jeremy Carroll 2013-05-06, 20:59
Bryan Beaudreault 2013-05-06, 21:44
Otis Gospodnetic 2013-05-06, 21:55
Bryan Beaudreault 2013-05-06, 21:59