Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase cluster over multiple EC2 Availability Zones?


Copy link to this message
-
Re: HBase cluster over multiple EC2 Availability Zones?
Yep, talking about HDFS replication.
On Mon, May 6, 2013 at 5:55 PM, Otis Gospodnetic <[EMAIL PROTECTED]
> wrote:

> Thanks Bryan and Jeremy, good info.
>
> When you say "replication", you are referring to HDFS block
> replication, not HBase cluster/table replication, correct?
>
> Thanks,
> Otis
> --
> Search Analytics - http://sematext.com/search-analytics/index.html
> HBASE Performance Monitoring - http://sematext.com/spm/index.html
>
>
>
>
>
> On Mon, May 6, 2013 at 5:44 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]> wrote:
> > We run multiple HBase clusters of various sizes, each is spread over 4
> AZs.
> >  Yes, there is a definite write latency added due to replication, and yes
> > there is a transfer cost that is not negligible but I can't comment on
> the
> > specifics of.
> >
> > Despite that though we use HBase as the primary datastore for multiple
> > online APIs and the sink for many hadoop jobs, and are able to keep total
> > response times plenty low for our apps.  Of course this is very specific
> to
> > our read/write patterns, etc, so YMMV.
> >
> >
> >
> >
> > On Mon, May 6, 2013 at 4:59 PM, Jeremy Carroll <[EMAIL PROTECTED]>
> wrote:
> >
> >> We tried it. It did not work well. According to AWS, cross AZ latency
> >> should be ~1ms. In practice we have seen large variances. Also if you
> are
> >> using larger instances with placement groups, they do not work cross AZ
> as
> >> well.
> >>
> >> With this in mind we went the multi-cluster route, and kept all nodes
> for a
> >> cluster in 1 AZ.
> >>
> >>
> >> On Mon, May 6, 2013 at 10:37 AM, Otis Gospodnetic <
> >> [EMAIL PROTECTED]> wrote:
> >>
> >> > Hi,
> >> >
> >> > Do people spread HBase clusters over multiple EC2 Availability Zones,
> >> > or is that a big no-no?  Cross-AZ communication is supposedly going
> >> > over fast fiber optic cables.
> >> >
> >> > Thanks,
> >> > Otis
> >> > --
> >> > Search Analytics - http://sematext.com/search-analytics/index.html
> >> > HBASE Performance Monitoring - http://sematext.com/spm/index.html
> >> >
> >>
>