Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Surprisingly high network traffic between kafka servers


Copy link to this message
-
Re: Surprisingly high network traffic between kafka servers
Carl, looking at the boundary chart it looks like you have periods of no
traffic also... prior to the spikes.

I also noticed you are using AWS from your logs, what instance types are
you using?  Do you have any network checks in place?

The logs show underReplication=true which leads towards what Joel was
theorizing as the issue.

Do you track stats on the cluster?
http://kafka.apache.org/documentation.html#monitoring I would expect
correlation of changes in the kafka stats and the boundary chart.

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Fri, Feb 7, 2014 at 2:47 AM, Carl Lerche <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB