Yeah totally that was what I thought you are saying. So what we have is a
yammer metrics which allows you to plug in any metrics reporter (it will do
jmx or whatever). But all this does it let you customize how the individual
java processes output metrics. We don't solve the problem of putting all
these together into a central dashboard. But these can be hooked up to a
standard monitoring thingy that pulls in metrics from all the java
processes in your environment (brokers, consumers, non-kafka related
things, etc) and let's you see these stats in aggregate across all machines.

I'm actually kind of ignorant of the state of open source monitoring
thingies because at linkedin we have a pretty badass in-house thing that
does this and is used for everything. I guess a lot of people used to use
ganglia or nagios for this but nowadays I hear all the cool kids use

At various times we have tossed around the idea of having a "Kafka
Monitoring Dashboard" that had a lot of stats on it. This would allow us to
make out-of-the-box monitoring a lot easier for Kafka. The feedback we have
gotten previously is that everyone has some inhouse tool they already use
for graphing/alerting/etc and they don't really want a one-off solution for
Kafka. This is definitely true for our own usage.


On Wed, Jul 24, 2013 at 10:04 AM, Otis Gospodnetic <
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB