Kafka, mail # user - Re: Keeping logs forever - 2013-02-22, 00:30
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Keeping logs forever
Anthony,

Is there a reason you wouldn't want to just push the data into something
built for cheap, long-term storage (like glacier, S3, or HDFS) and perhaps
"replay" from that instead of from the kafka brokers?  I can't speak for
Jay, Jun or Neha, but I believe the expected usage of Kafka is essentially
as a buffering mechanism to take the edge off the natural ebb-n-flow of
unpredictable internet traffic.  The highly available, long-term storage of
data is probably not at the top of their list of use cases when making
design decisions.

On Thu, Feb 21, 2013 at 6:00 PM, Anthony Grimes <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB