Home | About | Sematext search-lucene.com search-hadoop.com search-devops.com metrics + logs = try SPM and Logsene for free
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Metrics: via Broker vs. Producer vs. Consumer


+
Otis Gospodnetic 2013-07-24, 14:29
Copy link to this message
-
Re: Metrics: via Broker vs. Producer vs. Consumer
Yeah all our monitoring is just of the local process (basically just a
counter exposed through yammer metrics which support jmx and other
outputs). If I understand what you want instead of having a counter that
tracks, say, produce requests per second for a single broker you want one
that covers the whole cluster. Obviously this would require collecting the
local count and aggregating across all the brokers.

Our assumption is that you already have a separate monitoring system which
can slurp all these up, aggregate them, graph them, and alert off them.
There are a number of open source thingies like this and I think most
bigger shops have something they use. Our assumption is that trying to do a
kafka-specific monitoring system wouldn't work for most people because they
are wedded to their current setup and just want to integrate with that.

I'm not sure how valid any of those assumptions actually are.

-Jay
On Wed, Jul 24, 2013 at 7:29 AM, Otis Gospodnetic <
[EMAIL PROTECTED]> wrote:
 
+
Scott Clasen 2013-07-24, 16:25
+
Otis Gospodnetic 2013-07-24, 17:04
+
Jay Kreps 2013-07-24, 23:07
+
Otis Gospodnetic 2013-07-25, 09:35
+
Jun Rao 2013-07-26, 05:11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB