OK, I think I follow you.

Well, if the message volume is very low, then I don't think you need
the performance of Kafka. Perhaps a different design, where your
workers pull from a shared queue in memory somewhere might be better
(perhaps *that* queue could be filled by Kafka consumer reading from
an actual Kafka topic). Yes, the queue may need synchronization to
ensure each job only gets pulled off the queue once, but you said it's
low volume so performance shouldn't be a concern.


On Tue, Aug 13, 2013 at 7:13 PM, Eric Sites <[EMAIL PROTECTED]> wrote:

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB