Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HBase cluster over multiple EC2 Availability Zones?


+
Otis Gospodnetic 2013-05-06, 17:37
+
Amandeep Khurana 2013-05-06, 17:41
+
Otis Gospodnetic 2013-05-06, 17:58
+
Amandeep Khurana 2013-05-06, 18:00
+
Otis Gospodnetic 2013-05-06, 18:00
+
Amandeep Khurana 2013-05-06, 18:06
+
Mike Spreitzer 2013-05-06, 19:18
Copy link to this message
-
Re: HBase cluster over multiple EC2 Availability Zones?
Yes, I think so, but....  My worry is that AWS does suffer outages -
their AZs (== DCs) do get knocked out (at least the ones in N.
Virginia), so I figured is one can spread data over multiple AZs/DCs,
then the chances of the whole cluster being wiped out when 1 AZ goes
down will be lower.  Spreading things over N racks is good for
guarding against things like rack-specific equipment failures, so this
wouldn't help if the whole AZ/DC goes down.

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
HBASE Performance Monitoring - http://sematext.com/spm/index.html

On Mon, May 6, 2013 at 3:18 PM, Mike Spreitzer <[EMAIL PROTECTED]> wrote:
> If there were a way to ask for an HBase cluster to be spread across
> multiple racks within the same AZ, would this be valuable?
>
> Thanks,
> Mike
>
>
>
> From:   Amandeep Khurana <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>,
> Date:   05/06/2013 02:09 PM
> Subject:        Re: HBase cluster over multiple EC2 Availability Zones?
>
>
>
> Based on some testing I did on running MR on clusters across AZs, you take
> a 20-30% hit in performance purely due to latencies in replication. Of
> course, the numbers would vary based on the workload and my testing was
> definitely not comprehensive enough to cover all scenarios. This does not
> take into account the extra $$ you pay for going cross AZ, which as you
> can
> imagine would add up pretty quickly when looking at a large scale
> deployment.
>
>
> On Mon, May 6, 2013 at 11:00 AM, Otis Gospodnetic <
> [EMAIL PROTECTED]> wrote:
>
>> Re traffic and free vs. not free - Amandeep was right - according to
>> http://aws.amazon.com/ec2/pricing/#DataTransfer it's NOT free.  The
>> AWS person I recently spoke to about this told me it was free, but I
>> guess it's not!
>>
>> Otis
>> --
>> Search Analytics - http://sematext.com/search-analytics/index.html
>> HBASE Performance Monitoring - http://sematext.com/spm/index.html
>>
>>
>>
>>
>>
>> On Mon, May 6, 2013 at 1:58 PM, Otis Gospodnetic
>> <[EMAIL PROTECTED]> wrote:
>> > Hi,
>> >
>> > Actually, cross-AZ traffic is actually free.  It's cross-Region
>> > traffic that's not free.  I realize the cross-AZ (translates to
>> > cross-DC) traffic will be slower, but I wonder if there are any HBase
>> > (or HDFS)-specific reasons why one should not attempt to do this?
>> >
>> > Thanks,
>> > Otis
>> > --
>> > Search Analytics - http://sematext.com/search-analytics/index.html
>> > HBASE Performance Monitoring - http://sematext.com/spm/index.html
>> >
>> >
>> >
>> >
>> >
>> > On Mon, May 6, 2013 at 1:41 PM, Amandeep Khurana <[EMAIL PROTECTED]>
>> wrote:
>> >> I've not come across anyone spanning clusters cross AZ. You pay for
>> cross
>> >> AZ traffic and the link is slower than within a single AZ.
>> >>
>> >> Amandeep
>> >>
>> >>
>> >> On Mon, May 6, 2013 at 10:37 AM, Otis Gospodnetic <
>> >> [EMAIL PROTECTED]> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> Do people spread HBase clusters over multiple EC2 Availability
> Zones,
>> >>> or is that a big no-no?  Cross-AZ communication is supposedly going
>> >>> over fast fiber optic cables.
>> >>>
>> >>> Thanks,
>> >>> Otis
>> >>> --
>> >>> Search Analytics - http://sematext.com/search-analytics/index.html
>> >>> HBASE Performance Monitoring - http://sematext.com/spm/index.html
>> >>>
>>
>
+
Otis Gospodnetic 2013-05-06, 18:47
+
Steph Gosling 2013-05-06, 22:26
+
Andrew Purtell 2013-05-07, 00:01
+
Jeremy Carroll 2013-05-06, 20:59
+
Bryan Beaudreault 2013-05-06, 21:44
+
Otis Gospodnetic 2013-05-06, 21:55
+
Bryan Beaudreault 2013-05-06, 21:59