Kafka, mail # user - Re: Metrics: via Broker vs. Producer vs. Consumer - 2013-07-24, 17:04
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Plain View
Otis Gospodnetic 2013-07-24, 14:29
Jay Kreps 2013-07-24, 15:22
Scott Clasen 2013-07-24, 16:25
Copy link to this message
Re: Metrics: via Broker vs. Producer vs. Consumer

Ah, I think I didn't ask my question clearly.  Another try:
* If I have a javaagent attached to the Kafka process, I'll be able to connect to its JMX and get all the Broker metrics for that Broker process.
* If I have another Broker process, I'll need to attach my agent to this process, too, to get all Broker metrics associated with this second Broker process.
So far OK - like you said, I can sum, average, etc.

But what if I want to get all Producer metrics?  What do I need to do? I *believe* I would have to attach the javaagent to whichever app is acting as a Kafka Consumer and get Consumer stats from the JMX associated with the JVM process running that app.
Is this correct?
Is there any way to avoid that and get all Consumer and all Producer metrics using the javaagent attached to one of the Broker processes?

Performance Monitoring for Solr / ElasticSearch / HBase / Hadoop - http://sematext.com/spm 
Jay Kreps 2013-07-24, 23:07
Otis Gospodnetic 2013-07-25, 09:35
Jun Rao 2013-07-26, 05:11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB