Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Setting up kafka consumer to send logs to s3


Copy link to this message
-
Re: Setting up kafka consumer to send logs to s3
Quick search for "kafka s3 consumer" brings up a bunch of Github projects.
If you don't like any, I would write a kafka consumer in java that writes
to s3. Probably less than 200 lines of code.

F.

On 6/25/13 1:50 PM, "Alan Everdeen" <[EMAIL PROTECTED]> wrote:

>In my application, I have logs that are sent as kafka messages, and I need
>a way to save these logs to an s3 bucket using a kafka sink.
>
>I was wondering what the recommended approach would be to accomplish this
>task, assuming I am using Kafka 0.8.
>
>
>Thank you for your time,
>Alan Everdeen
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB