It pushes metrics to Riemann (http://riemann.io) a very cool tool for aggregating and processing event streams. We've experimented with it a little in the past but this is the first production integration we've had. Streams are processed by Riemann and then plotted by Librato ( http://librato.com).
Installing requires building the project and copying the JAR to Kafka's ./libs dir and then putting the following in the server.properties:
It lets you push directly from the broker to a Riemann service. However, we actually push messages (the Riemann events- protocol buffer messages) onto a Kafka topic to be later consumed by a bridged and forwarded to Riemann. This has the added benefit of allowing metrics to be flowed into our existing Redshift data load pipeline as well as being monitored in realtime.
It's pretty early days but I deployed it to our small Kafka cluster on EC2 earlier and have been watching the graphs all day. Although we were writing metrics to CSVs before we had nothing to visualise it, this makes it pretty easy to pump the broker metrics out.
I thought I'd mention it here in case other people are in a similar position to us and wanted an easy (alebit still a little experimental :) way to get the metrics out of Kafka.
I'd really appreciate some feedback, and of course I'd be delighted if other people use it and help fix my mistakes :)
At the moment it's built against an earlier 0.8 that we're running in production (9bd2a) but I'm keen to start working (and deploying) an officially released build.
Jun On Thu, Jul 18, 2013 at 2:00 PM, Paul Ingles <[EMAIL PROTECTED]> wrote:
Jun Rao 2013-07-22, 05:37
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation projects and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext