Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka on EC2

Copy link to this message
Re: Kafka on EC2
>> Single m1.large instance's capacity was 10k/sec.

When you say capacity, did you mean the I/O or network capacity on the
m1.large instances ?

On Sun, Nov 18, 2012 at 3:26 PM, Bae, Jae Hyeon <[EMAIL PROTECTED]> wrote:
> I am running kafka on ec2 with m1.large instance. I think that large
> number of low end servers will outperform small number of high end
> servers, but I am not sure I am correct.
> I assumed 12 m1.large instances would be able to handle more than 6
> billion rows in a day but my expectation was wrong. Single m1.large
> instance's capacity was 10k/sec. If we want to have a room, we'd
> better assume kafka on m1.large maximum capacity is 8k/sec.
> On Fri, Nov 16, 2012 at 5:57 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:
>> At least based on prior threads (discussing experiences/issues with ec2),
>> there should be a number of people on this list who can help you. It would
>> be helpful if we have a ec2-operations wiki @
>> https://cwiki.apache.org/confluence/display/KAFKA/Index . Would people be
>> interested in sharing operational experiences there? I'm thinking something
>> similar to the operations wiki that's already available (
>> https://cwiki.apache.org/confluence/display/KAFKA/Operations).
>> Joel
>> On Fri, Nov 16, 2012 at 5:04 PM, Senthilvel Rangaswamy <[EMAIL PROTECTED]
>>> wrote:
>>> Have folks implemented large installations on Kafka on Amazon EC2. I am
>>> looking for best practices. Like the kind of nodes, EBS vs Instance store
>>> etc.,
>>> --
>>> ..Senthil
>>> "If there's anything more important than my ego around, I want it
>>>  caught and shot now."
>>>                                                     - Douglas Adams.