Another idea. If a set of messages arrive over a single TCP connection, route to a partition depending on TCP connection.

To be honest, these approaches, while they work, may not scale when the message rate is high. If at all possible, try to think of a way to remove this requirement from your design. For example, a design might have a sequence number assigned to each message before it goes into Kafka (a time-based UUID, for example), and something later in the pipe line sorts it all out. Kafka then does what is does best, IMHO, a high-performance reliable, message bus.


On Jun 14, 2013, at 7:37 AM, David Arthur <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB