Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HBase cluster over multiple EC2 Availability Zones?


+
Otis Gospodnetic 2013-05-06, 17:37
+
Amandeep Khurana 2013-05-06, 17:41
+
Otis Gospodnetic 2013-05-06, 17:58
+
Amandeep Khurana 2013-05-06, 18:00
+
Otis Gospodnetic 2013-05-06, 18:00
+
Amandeep Khurana 2013-05-06, 18:06
+
Mike Spreitzer 2013-05-06, 19:18
+
Otis Gospodnetic 2013-05-06, 20:13
+
Otis Gospodnetic 2013-05-06, 18:47
+
Steph Gosling 2013-05-06, 22:26
+
Andrew Purtell 2013-05-07, 00:01
+
Jeremy Carroll 2013-05-06, 20:59
+
Bryan Beaudreault 2013-05-06, 21:44
Copy link to this message
-
Re: HBase cluster over multiple EC2 Availability Zones?
Thanks Bryan and Jeremy, good info.

When you say "replication", you are referring to HDFS block
replication, not HBase cluster/table replication, correct?

Thanks,
Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
HBASE Performance Monitoring - http://sematext.com/spm/index.html

On Mon, May 6, 2013 at 5:44 PM, Bryan Beaudreault
<[EMAIL PROTECTED]> wrote:
> We run multiple HBase clusters of various sizes, each is spread over 4 AZs.
>  Yes, there is a definite write latency added due to replication, and yes
> there is a transfer cost that is not negligible but I can't comment on the
> specifics of.
>
> Despite that though we use HBase as the primary datastore for multiple
> online APIs and the sink for many hadoop jobs, and are able to keep total
> response times plenty low for our apps.  Of course this is very specific to
> our read/write patterns, etc, so YMMV.
>
>
>
>
> On Mon, May 6, 2013 at 4:59 PM, Jeremy Carroll <[EMAIL PROTECTED]> wrote:
>
>> We tried it. It did not work well. According to AWS, cross AZ latency
>> should be ~1ms. In practice we have seen large variances. Also if you are
>> using larger instances with placement groups, they do not work cross AZ as
>> well.
>>
>> With this in mind we went the multi-cluster route, and kept all nodes for a
>> cluster in 1 AZ.
>>
>>
>> On Mon, May 6, 2013 at 10:37 AM, Otis Gospodnetic <
>> [EMAIL PROTECTED]> wrote:
>>
>> > Hi,
>> >
>> > Do people spread HBase clusters over multiple EC2 Availability Zones,
>> > or is that a big no-no?  Cross-AZ communication is supposedly going
>> > over fast fiber optic cables.
>> >
>> > Thanks,
>> > Otis
>> > --
>> > Search Analytics - http://sematext.com/search-analytics/index.html
>> > HBASE Performance Monitoring - http://sematext.com/spm/index.html
>> >
>>
+
Bryan Beaudreault 2013-05-06, 21:59