Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - HBase cluster over multiple EC2 Availability Zones?


+
Otis Gospodnetic 2013-05-06, 17:37
+
Amandeep Khurana 2013-05-06, 17:41
+
Otis Gospodnetic 2013-05-06, 17:58
+
Amandeep Khurana 2013-05-06, 18:00
+
Otis Gospodnetic 2013-05-06, 18:00
+
Amandeep Khurana 2013-05-06, 18:06
+
Mike Spreitzer 2013-05-06, 19:18
+
Otis Gospodnetic 2013-05-06, 20:13
+
Otis Gospodnetic 2013-05-06, 18:47
+
Steph Gosling 2013-05-06, 22:26
Copy link to this message
-
Re: HBase cluster over multiple EC2 Availability Zones?
Andrew Purtell 2013-05-07, 00:01
> We do simple 'rack' awareness based on the AZ returned by the metadata
server

Just want to add a datapoint that I do this too -- for simulating racks
when running test clusters up on EC2 (it corresponds to physical properties
and makes it easy to group instances for nuking a whole "rack") -- and have
encountered no major problems. I can also echo the comments of others that
cross-AZ deployments can see significant delay and variance on the network.
On Mon, May 6, 2013 at 3:26 PM, Steph Gosling <[EMAIL PROTECTED]> wrote:

> Hi,
>
> On Mon, 6 May 2013 14:47:49 -0400
> Otis Gospodnetic <[EMAIL PROTECTED]> wrote:
>
> > Yup.  I'm suddenly turned off by that penny per GB! :)
> > Ignoring the dollars, it sounds like one would just have to be OK with
> > increased latencies, but technically nothing would break.  Doodling
> > our architecture on paper here, I think we may as well just have
> > complete, independent setups in multiple Regions then - I suspect
> > those pennies add up faster than one would think.
> >
> > Otis
>
> I run a couple of small HBase clusters (low double-digit nodes each),
> and both span AZs in their respective regions (we're not doing any
> inter-region stuff yet, nor do I expect to,TBH). What AWS don't tell
> you is that not all instance types are available in all AZs,
> particularly for the newer or more esoteric instances.
>
> We care about this data so any performace hit (not that we've
> particularly noticed one) because of cross AZ traffic is acceptable. We
> do simple 'rack' awareness based on the AZ returned by the metadata
> server, you could probably fine-tune that based on subnet if your
> cluster got big but we've not had the need to.
>
> With regards to performance specifically, I've not looked explicitly
> but I'd expect that you'll see far more variance based on things like
> instance size, your neighbours on the same host and their behaviour.
>
> Finally, I'm also surprised about the inter-AZ data charges, that seems
> to be a very wide-spread misconception, and yeah I'd imagine the
> pennies do add up...
>
> Cheers,
>
> Steph
> --
> Steph Gosling <[EMAIL PROTECTED]>
>

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
+
Jeremy Carroll 2013-05-06, 20:59
+
Bryan Beaudreault 2013-05-06, 21:44
+
Otis Gospodnetic 2013-05-06, 21:55
+
Bryan Beaudreault 2013-05-06, 21:59