Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # user >> EC2 disk configuration for zookeeper? One big disk or two smaller ones per node?


Copy link to this message
-
RE: EC2 disk configuration for zookeeper? One big disk or two smaller ones per node?
I'd also add that EC2 supports a variety of different ways of configuring the disks which should be explored if you are very write heavy. They include:
* Using high IOPS instances
* Using multiple disks and doing OS level RAID across them to present a single logical disk to the app
* Local vs. EBS volumes
* Standard vs. provisioned IOPS w/EBS

I'd be interested to know if others have already done benchmarking of these different configurations. I've never heard about it if so (but I’m pretty new to ZK so that doesn't mean much :)).

~Eric

-----Original Message-----
From: Ted Dunning [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, October 09, 2012 8:29 AM
To: [EMAIL PROTECTED]
Subject: Re: EC2 disk configuration for zookeeper? One big disk or two smaller ones per node?

Two disks could be a significant advantage.

You should also experiment with ways to avoid VM induced gaps in time.

Finally, if you really are going to be write heavy, 3 nodes are likely to perform better than 5.

On Tue, Oct 9, 2012 at 2:03 PM, Brian Tarbox <[EMAIL PROTECTED]> wrote:

> In setting up an Amazon EC2 configuration for a group of zookeeper
> nodes I'm wondering about disk configurations.  The zk documentation
> stresses the advantages of a dedicated disk for the transaction log.  
> Does this imply that two smaller disks would be more performant than a single larger disk?
>  I'm anticipating a 5 node write-heavy cluster.
>
> Thanks.
>
> --
> http://about.me/BrianTarbox
>