Kafka, mail # user - Re: Surprisingly high network traffic between kafka servers - 2014-02-07, 13:43
 Search Hadoop and all its subprojects:

Switch to Plain View
+
zhong dong 2014-02-14, 06:31
+
Jun Rao 2014-02-14, 15:55
+
Jay Kreps 2014-02-14, 17:25
+
Carl Lerche 2014-02-15, 00:11
+
Carl Lerche 2014-02-05, 04:46
+
Guozhang Wang 2014-02-05, 05:26
+
Jay Kreps 2014-02-05, 05:37
+
Carl Lerche 2014-02-05, 05:52
+
S Ahmed 2014-02-05, 17:07
+
Carl Lerche 2014-02-05, 19:08
+
Jay Kreps 2014-02-05, 20:31
+
Carl Lerche 2014-02-06, 00:51
+
Joel Koshy 2014-02-06, 01:18
+
Neha Narkhede 2014-02-06, 12:06
+
Carl Lerche 2014-02-07, 03:49
+
Neha Narkhede 2014-02-07, 04:25
+
Jun Rao 2014-02-07, 05:06
+
Carl Lerche 2014-02-07, 07:18
+
Carl Lerche 2014-02-07, 07:44
+
Carl Lerche 2014-02-07, 07:47
Copy link to this message
-
Re: Surprisingly high network traffic between kafka servers
Carl, looking at the boundary chart it looks like you have periods of no
traffic also... prior to the spikes.

I also noticed you are using AWS from your logs, what instance types are
you using?  Do you have any network checks in place?

The logs show underReplication=true which leads towards what Joel was
theorizing as the issue.

Do you track stats on the cluster?
http://kafka.apache.org/documentation.html#monitoring I would expect
correlation of changes in the kafka stats and the boundary chart.

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Fri, Feb 7, 2014 at 2:47 AM, Carl Lerche <[EMAIL PROTECTED]> wrote:
 
+
Carl Lerche 2014-02-07, 21:29
+
Dong Zhong 2014-02-19, 11:46
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB