Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - resilient producer


+
Stan Rosenberg 2013-01-15, 19:29
Copy link to this message
-
Re: resilient producer
Corbin Hoenes 2013-01-15, 20:13
+1 how about posting yours to GitHub?
Sounds like a good contrib project.

Sent from my iPhone

On Jan 15, 2013, at 12:29 PM, Stan Rosenberg <[EMAIL PROTECTED]> wrote:

> Hi,
>
> In out current data ingestion system, producers are resilient in the sense
> that if data cannot be reliably published (e.g., network is down), it is
> spilled onto local disk.
> A separate process runs asynchronously and attempts to publish spilled
> data.  I am curious to hear what other people do in this case.
> Is there a plan to have something similar integrated into kafka?  (AFAIK,
> current implementation gives up after a configurable number of retries.)
>
> Thanks,
>
> stan

 
+
Stan Rosenberg 2013-01-15, 22:32
+
Jay Kreps 2013-01-15, 20:19
+
Stan Rosenberg 2013-01-15, 22:30