Kafka, mail # user - Re: How to design a robust producer? - 2014-01-30, 14:38
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
Re: How to design a robust producer?
Well, you could start by looking at the Kafka Producer source code for some
ideas. We have built plenty of solid software on that.

As to your goal of building something solid, robust, and critical. All I
can say is you then need to keep your Producer as simple as possible -- the
simpler it is, the less like it is to crash, have bugs, and you must test
it very well. Get the data to Kafka as fast as possible, so the chance of
losing any due to a crash are very small. Take a long time to test it. The
Producers I have written (in C++) run for weeks without going down (and
then we usually bring them down on purpose for upgrades). However, they
were in test for months too.

On Thu, Jan 30, 2014 at 6:31 AM, Thibaud Chardonnens
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB