“unless you have a good reason to load balance and manage offsets manually”
An alternative to using simpleconsumer in this use case is to use the
zookeeper consumer connector and turn off auto commit. After your consumer
process is done processing a batch of messages you can all commitOffsets -
the main caveat to be aware of is that if your consumer processes batches
very fast you would write to zookeeper that often - so in fact setting an
autocommit interval and being willing to deal with duplicates is almost
equivalent. KAFKA-657 would help I think - since once that API is available
you can store your offsets anywhere you like.


NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB