Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - EC2 instance type recommendation ?


Copy link to this message
-
Re: EC2 instance type recommendation ?
Bryan Beaudreault 2013-07-16, 18:31
You're right that EC2 does not provide great instance types for HBase.  In
our research we found that most people seemed to be using c1.xlarge.  This
still is not great, because 7GB of RAM for DataNode + RegionServer + OS is
pitiful, but it works.  Of course that also makes it impossible to colocate
TaskTrackers on the RegionServer nodes.  We also tried m1.xlarge in the
beginning, and it's nice to have the memory but we quickly became CPU bound
and thus moved to c1.xlarge.  Of course you're not going to get good disk
performance on these ephemeral disks, but EBS is trouble in our experience.

At HBaseCon, the pinterest engineers mentioned using hi1.4xlarge.  Those
are very expensive though and unfortunately just have 2 SSD disks.  Perhaps
they have benchmarks or can chime in as to whether they ventured to use EBS
to supplement the 2 disks or what.
On Tue, Jul 16, 2013 at 2:18 PM, Amit Mor <[EMAIL PROTECTED]> wrote:

> Hello, I am curious to hear the recommendations people have here for
> running HBase on EC2 instances. I failed to find an instance that has a
> good ratio/balance between CPU, # of disks and jvm acceptable heap Memory
> to be used with (almost random) read intensive cluster. The major
> bottleneck I found was the disk read latency that could even reach srvtm of
> a second ... Combined with everything else ;-)
>
> Thanks,
> Amit
>