Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase cluster over multiple EC2 Availability Zones?


Copy link to this message
-
Re: HBase cluster over multiple EC2 Availability Zones?
Yup.  I'm suddenly turned off by that penny per GB! :)
Ignoring the dollars, it sounds like one would just have to be OK with
increased latencies, but technically nothing would break.  Doodling
our architecture on paper here, I think we may as well just have
complete, independent setups in multiple Regions then - I suspect
those pennies add up faster than one would think.

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
HBASE Performance Monitoring - http://sematext.com/spm/index.html

On Mon, May 6, 2013 at 2:06 PM, Amandeep Khurana <[EMAIL PROTECTED]> wrote:
> Based on some testing I did on running MR on clusters across AZs, you take
> a 20-30% hit in performance purely due to latencies in replication. Of
> course, the numbers would vary based on the workload and my testing was
> definitely not comprehensive enough to cover all scenarios. This does not
> take into account the extra $$ you pay for going cross AZ, which as you can
> imagine would add up pretty quickly when looking at a large scale
> deployment.
>
>
> On Mon, May 6, 2013 at 11:00 AM, Otis Gospodnetic <
> [EMAIL PROTECTED]> wrote:
>
>> Re traffic and free vs. not free - Amandeep was right - according to
>> http://aws.amazon.com/ec2/pricing/#DataTransfer it's NOT free.  The
>> AWS person I recently spoke to about this told me it was free, but I
>> guess it's not!
>>
>> Otis
>> --
>> Search Analytics - http://sematext.com/search-analytics/index.html
>> HBASE Performance Monitoring - http://sematext.com/spm/index.html
>>
>>
>>
>>
>>
>> On Mon, May 6, 2013 at 1:58 PM, Otis Gospodnetic
>> <[EMAIL PROTECTED]> wrote:
>> > Hi,
>> >
>> > Actually, cross-AZ traffic is actually free.  It's cross-Region
>> > traffic that's not free.  I realize the cross-AZ (translates to
>> > cross-DC) traffic will be slower, but I wonder if there are any HBase
>> > (or HDFS)-specific reasons why one should not attempt to do this?
>> >
>> > Thanks,
>> > Otis
>> > --
>> > Search Analytics - http://sematext.com/search-analytics/index.html
>> > HBASE Performance Monitoring - http://sematext.com/spm/index.html
>> >
>> >
>> >
>> >
>> >
>> > On Mon, May 6, 2013 at 1:41 PM, Amandeep Khurana <[EMAIL PROTECTED]>
>> wrote:
>> >> I've not come across anyone spanning clusters cross AZ. You pay for
>> cross
>> >> AZ traffic and the link is slower than within a single AZ.
>> >>
>> >> Amandeep
>> >>
>> >>
>> >> On Mon, May 6, 2013 at 10:37 AM, Otis Gospodnetic <
>> >> [EMAIL PROTECTED]> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> Do people spread HBase clusters over multiple EC2 Availability Zones,
>> >>> or is that a big no-no?  Cross-AZ communication is supposedly going
>> >>> over fast fiber optic cables.
>> >>>
>> >>> Thanks,
>> >>> Otis
>> >>> --
>> >>> Search Analytics - http://sematext.com/search-analytics/index.html
>> >>> HBASE Performance Monitoring - http://sematext.com/spm/index.html
>> >>>
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB