Carl, looking at the boundary chart it looks like you have periods of no
traffic also... prior to the spikes.

I also noticed you are using AWS from your logs, what instance types are
you using?  Do you have any network checks in place?

The logs show underReplication=true which leads towards what Joel was
theorizing as the issue.

Do you track stats on the cluster? I would expect
correlation of changes in the kafka stats and the boundary chart.

 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 Twitter: @allthingshadoop <>
On Fri, Feb 7, 2014 at 2:47 AM, Carl Lerche <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB