thinking/typing out loud here not sure if this is the problem but could be
so figure I throw it out there

the ZookeeperConsumerConnector has a messageStreamCreated atomic boolean
stopping more than one consumer connector being created for messages stream
by filter at once...

do you have separate instantiated objects of your ConsumerConnector (so
calling ConsumerConnector.create for each one) or do you have one
ConsumerConnector.create and then you call createMessageStreams multiple
times on it (which would cause a RuntimeException so probably not but maybe
your code is swallowing that)... if the latter that could explain why it is
lagging behind unexpectedly since really one ConsumerConnector is

again, you might not be doing this but figure I throw it out there in case
you were.

 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 Twitter: @allthingshadoop <>
On Wed, Jan 8, 2014 at 1:44 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB