You can try to reduce batch.num.messages first and see if the throughput
gets affected.

For your second question, we do not have a good solution to that since the
offsets are not consistent across data centers like you said. One way we
did is to have the consumer consuming both data centers, but keep the slave
data center's consumer do no-op; once the master data center is totally
gone, turn the nob to let the slave data center's consumers to both consume
and process. Doing so still do not give you consistency as you may consume
duplicates or data loss, but gives you a relatively close point of resuming
compared with the previous consumer.

On Mon, Jun 9, 2014 at 2:08 PM, Kane Kane <[EMAIL PROTECTED]> wrote:

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB