Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka on EC2


Copy link to this message
-
Re: Kafka on EC2
I'd only consider m1.xlarge and higher for Kafka. The m1.xlarge have "high" I/O performance according to Amazon. This is disk I/O and network I/O performance. Of course you need to use EBS volumes if you want your Kafka brokers to survive reboots - you can expect reboots on AWS. Some people have reported I/O improvements by RAIDing EBS volumes (http://alestic.com/2009/06/ec2-ebs-raid). Deploying in the same region as your application will also improve performance.
On Nov 16, 2012, at 8:04 PM, Senthilvel Rangaswamy wrote:

> Have folks implemented large installations on Kafka on Amazon EC2. I am
> looking for best practices. Like the kind of nodes, EBS vs Instance store
> etc.,
>
> --
> ..Senthil
>
> "If there's anything more important than my ego around, I want it
> caught and shot now."
>                                                    - Douglas Adams.