Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase cluster over multiple EC2 Availability Zones?


Copy link to this message
-
Re: HBase cluster over multiple EC2 Availability Zones?
Yep, talking about HDFS replication.
On Mon, May 6, 2013 at 5:55 PM, Otis Gospodnetic <[EMAIL PROTECTED]
> wrote:

> Thanks Bryan and Jeremy, good info.
>
> When you say "replication", you are referring to HDFS block
> replication, not HBase cluster/table replication, correct?
>
> Thanks,
> Otis
> --
> Search Analytics - http://sematext.com/search-analytics/index.html
> HBASE Performance Monitoring - http://sematext.com/spm/index.html
>
>
>
>
>
> On Mon, May 6, 2013 at 5:44 PM, Bryan Beaudreault
> <[EMAIL PROTECTED]> wrote:
> > We run multiple HBase clusters of various sizes, each is spread over 4
> AZs.
> >  Yes, there is a definite write latency added due to replication, and yes
> > there is a transfer cost that is not negligible but I can't comment on
> the
> > specifics of.
> >
> > Despite that though we use HBase as the primary datastore for multiple
> > online APIs and the sink for many hadoop jobs, and are able to keep total
> > response times plenty low for our apps.  Of course this is very specific
> to
> > our read/write patterns, etc, so YMMV.
> >
> >
> >
> >
> > On Mon, May 6, 2013 at 4:59 PM, Jeremy Carroll <[EMAIL PROTECTED]>
> wrote:
> >
> >> We tried it. It did not work well. According to AWS, cross AZ latency
> >> should be ~1ms. In practice we have seen large variances. Also if you
> are
> >> using larger instances with placement groups, they do not work cross AZ
> as
> >> well.
> >>
> >> With this in mind we went the multi-cluster route, and kept all nodes
> for a
> >> cluster in 1 AZ.
> >>
> >>
> >> On Mon, May 6, 2013 at 10:37 AM, Otis Gospodnetic <
> >> [EMAIL PROTECTED]> wrote:
> >>
> >> > Hi,
> >> >
> >> > Do people spread HBase clusters over multiple EC2 Availability Zones,
> >> > or is that a big no-no?  Cross-AZ communication is supposedly going
> >> > over fast fiber optic cables.
> >> >
> >> > Thanks,
> >> > Otis
> >> > --
> >> > Search Analytics - http://sematext.com/search-analytics/index.html
> >> > HBASE Performance Monitoring - http://sematext.com/spm/index.html
> >> >
> >>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB