Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Kafka on EC2


+
Senthilvel Rangaswamy 2012-11-17, 01:04
+
Joel Koshy 2012-11-17, 01:57
+
Bae, Jae Hyeon 2012-11-18, 23:26
+
Neha Narkhede 2012-11-18, 23:36
+
Bae, Jae Hyeon 2012-11-20, 00:36
+
David Arthur 2012-11-20, 15:52
+
Evan Chan 2012-11-20, 17:12
Copy link to this message
-
Re: Kafka on EC2
I'd only consider m1.xlarge and higher for Kafka. The m1.xlarge have "high" I/O performance according to Amazon. This is disk I/O and network I/O performance. Of course you need to use EBS volumes if you want your Kafka brokers to survive reboots - you can expect reboots on AWS. Some people have reported I/O improvements by RAIDing EBS volumes (http://alestic.com/2009/06/ec2-ebs-raid). Deploying in the same region as your application will also improve performance.
On Nov 16, 2012, at 8:04 PM, Senthilvel Rangaswamy wrote:

> Have folks implemented large installations on Kafka on Amazon EC2. I am
> looking for best practices. Like the kind of nodes, EBS vs Instance store
> etc.,
>
> --
> ..Senthil
>
> "If there's anything more important than my ego around, I want it
> caught and shot now."
>                                                    - Douglas Adams.

+
Matthew Rathbone 2012-11-19, 19:44
+
James A. Robinson 2012-11-19, 19:56
+
Matt Jones 2012-11-19, 21:34
+
Matthew Rathbone 2012-11-19, 21:44
+
Senthilvel Rangaswamy 2012-11-19, 20:17