Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Surprisingly high network traffic between kafka servers


+
Carl Lerche 2014-02-05, 04:46
+
Guozhang Wang 2014-02-05, 05:26
+
Jay Kreps 2014-02-05, 05:37
+
Carl Lerche 2014-02-05, 05:52
+
Jay Kreps 2014-02-05, 20:31
+
Carl Lerche 2014-02-06, 00:51
+
Joel Koshy 2014-02-06, 01:18
+
Neha Narkhede 2014-02-06, 12:06
+
Carl Lerche 2014-02-07, 03:49
+
Neha Narkhede 2014-02-07, 04:25
+
Jun Rao 2014-02-07, 05:06
+
Carl Lerche 2014-02-07, 07:44
+
Carl Lerche 2014-02-07, 07:47
Copy link to this message
-
Re: Surprisingly high network traffic between kafka servers
Carl, looking at the boundary chart it looks like you have periods of no
traffic also... prior to the spikes.

I also noticed you are using AWS from your logs, what instance types are
you using?  Do you have any network checks in place?

The logs show underReplication=true which leads towards what Joel was
theorizing as the issue.

Do you track stats on the cluster?
http://kafka.apache.org/documentation.html#monitoring I would expect
correlation of changes in the kafka stats and the boundary chart.

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Fri, Feb 7, 2014 at 2:47 AM, Carl Lerche <[EMAIL PROTECTED]> wrote:
 
+
Carl Lerche 2014-02-07, 21:29
+
Carl Lerche 2014-02-07, 07:18
+
S Ahmed 2014-02-05, 17:07
+
Carl Lerche 2014-02-05, 19:08
+
zhong dong 2014-02-14, 06:31
+
Carl Lerche 2014-02-15, 00:11
+
Jun Rao 2014-02-14, 15:55
+
Jay Kreps 2014-02-14, 17:25
+
Dong Zhong 2014-02-19, 11:46
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB