Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Amazon SNS and Kafka comparison


Copy link to this message
-
Re: Amazon SNS and Kafka comparison
Depends how important being able to access every single bit of the messages are, right down to looking at what is on the disk. It's very important to us, we need that control. Ability to scale throughout as needed is also important - too important to do anything but run it ourselves.  All these reasons meant we use Kafka.

As for operational overhead we have found Kafka 0.72 to be very stable and easy to manage. If *only* other components were as easy. :-)

Philip

On Jun 14, 2013, at 7:45 AM, James Newhaven <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I have a system that needs to process tens of thousands of user events per
> second.
>
> I've looked at both Kafka and Amazon SNS.
>
> Using SNS would mean I can avoid the operational overhead of maintaining
> Kafka and Zookeeper installations and monitoring. I also wouldn't need to
> worry about storage and network infrastructure.
>
> On the other hand, SNS  could be very costly, particularly since I must
> also use SQS to persist messages until consumed.
>
> Has anyone else considered using Amazon SNS instead of Kafka? I would be
> interested to hear people's opinions on both.
>
> Thanks,
> James

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB