Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> resilient producer


+
Stan Rosenberg 2013-01-15, 19:29
Copy link to this message
-
Re: resilient producer
+1 how about posting yours to GitHub?
Sounds like a good contrib project.

Sent from my iPhone

On Jan 15, 2013, at 12:29 PM, Stan Rosenberg <[EMAIL PROTECTED]> wrote:

> Hi,
>
> In out current data ingestion system, producers are resilient in the sense
> that if data cannot be reliably published (e.g., network is down), it is
> spilled onto local disk.
> A separate process runs asynchronously and attempts to publish spilled
> data.  I am curious to hear what other people do in this case.
> Is there a plan to have something similar integrated into kafka?  (AFAIK,
> current implementation gives up after a configurable number of retries.)
>
> Thanks,
>
> stan

 
+
Stan Rosenberg 2013-01-15, 22:32
+
Jay Kreps 2013-01-15, 20:19
+
Stan Rosenberg 2013-01-15, 22:30
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB