Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka on EC2


Copy link to this message
-
Re: Kafka on EC2
>> Single m1.large instance's capacity was 10k/sec.

When you say capacity, did you mean the I/O or network capacity on the
m1.large instances ?

Thanks,
Neha
On Sun, Nov 18, 2012 at 3:26 PM, Bae, Jae Hyeon <[EMAIL PROTECTED]> wrote:
> I am running kafka on ec2 with m1.large instance. I think that large
> number of low end servers will outperform small number of high end
> servers, but I am not sure I am correct.
>
> I assumed 12 m1.large instances would be able to handle more than 6
> billion rows in a day but my expectation was wrong. Single m1.large
> instance's capacity was 10k/sec. If we want to have a room, we'd
> better assume kafka on m1.large maximum capacity is 8k/sec.
>
> On Fri, Nov 16, 2012 at 5:57 PM, Joel Koshy <[EMAIL PROTECTED]> wrote:
>> At least based on prior threads (discussing experiences/issues with ec2),
>> there should be a number of people on this list who can help you. It would
>> be helpful if we have a ec2-operations wiki @
>> https://cwiki.apache.org/confluence/display/KAFKA/Index . Would people be
>> interested in sharing operational experiences there? I'm thinking something
>> similar to the operations wiki that's already available (
>> https://cwiki.apache.org/confluence/display/KAFKA/Operations).
>>
>> Joel
>>
>>
>> On Fri, Nov 16, 2012 at 5:04 PM, Senthilvel Rangaswamy <[EMAIL PROTECTED]
>>> wrote:
>>
>>> Have folks implemented large installations on Kafka on Amazon EC2. I am
>>> looking for best practices. Like the kind of nodes, EBS vs Instance store
>>> etc.,
>>>
>>> --
>>> ..Senthil
>>>
>>> "If there's anything more important than my ego around, I want it
>>>  caught and shot now."
>>>                                                     - Douglas Adams.
>>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB