Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> JMXTrans not sending kafka 0.8 metrics to Ganglia


+
Priya Matpadi 2013-11-05, 01:02
+
Paul Mackles 2013-11-05, 01:11
+
Priya Matpadi 2013-11-05, 06:00
+
Paul Mackles 2013-11-05, 20:36
Copy link to this message
-
Re: JMXTrans not sending kafka 0.8 metrics to Ganglia
typeNames is required in settings section for beanName to be included in
the metric name when jmxtrans sends it out. Here's an example:

      "outputWriters" : [ {
        "@class" :
"com.googlecode.jmxtrans.model.output.GangliaWriter",
        "settings" : {
          "groupName" : "kafka",
          "port" : 8649,
          "host" : "hostname",
          "typeNames" : [ "name" ]
        }
      } ],
      "obj": "\"kafka.server\":type=\"BrokerTopicMetrics\",name=\"*\"",
      "attr": [
      "Count",
      "FifteenMinuteRate",
      "FiveMinuteRate",
      "OneMinuteRate",
      "MeanRate"
      ],
      "resultAlias": "BrokerTopicMetrics"

    } ]
On Tue, Nov 5, 2013 at 12:35 PM, Paul Mackles <[EMAIL PROTECTED]> wrote:

> In our case, jmxtrans returns:
>
> ReplicaManager.ISRShrinksPerSec.Count
> ReplicaManager.ISRShrinksPerSec.MeanRate
>
> ReplicaManager.ISRExpandsPerSec.OneMinuteRate
>
> ReplicaManager.ISRExpandsPerSec.Count
> ReplicaManager.ISRExpandsPerSec.MeanRate
> ReplicaManager.ISRExpandsPerSec.OneMinuteRate
> ReplicaManager.LeaderCount.Value
>
> ReplicaManager.PartitionCount.Value
>
>
>
> In other words, the bean name is include in the metric name when jmxtrans
> sends it out. In our case, we are using graphite. Not sure if that makes a
> difference.
>
> Paul
>
> On 11/5/13 12:59 AM, "Priya Matpadi" <[EMAIL PROTECTED]> wrote:
>
> >Paul,
> >Thank you! I didn't realize the quotes are important to keep.
> >I have question on name=\"*\""
> >
> >For the type=ReplicaManager, there are at least 4 Mbean names
> >"kafka.server":name="PartitionCount",type="ReplicaManager"
> >"kafka.server":name="LeaderCount",type="ReplicaManager"
> >
> >"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> >"kafka.server":name="ISRExpandsPerSec",type="ReplicaManager"
> >
> >In ganglia metrics dropdown, I only see 4 elements as configured above
> >i.e.
> >
> >ReplicaManager.Count
> >ReplicaManager.OneMinuteRate
> >ReplicaManager.MeanRate
> >ReplicaManager.Value
> >
> >Then, how can one tell if for e.g. Value pertains to LeaderCount or
> >PartitionCount?
> >Even though it is laborious, isn't it good idea to explicitly specify the
> >name?
> >
> >Thanks,
> >Priya
> >
> >
> >
> >
> >On Mon, Nov 4, 2013 at 5:10 PM, Paul Mackles <[EMAIL PROTECTED]> wrote:
> >
> >> It looks like you are missing quotes in the object name. Here is a
> >>snippet
> >> from our jmxtrans configs:
> >>
> >> "resultAlias": "ReplicaManager",
> >>      "obj": "\"kafka.server\":type=\"ReplicaManager\",name=\"*\"",
> >>      "attr": [
> >>       "Count",
> >>       "OneMinuteRate",
> >>       "MeanRate",
> >>       "Value"
> >>      ]
> >>
> >>
> >> Unless more recent versions of kafka get rid of the quotes (we are on an
> >> older 0.8 version still).
> >>
> >> Paul
> >>
> >> On 11/4/13 8:02 PM, "Priya Matpadi" <[EMAIL PROTECTED]>
> wrote:
> >>
> >> >I am trying to send kafka metrics for display to ganglia server using
> >> >latest download from https://github.com/adambarthelson/kafka-ganglia.
> >> >
> >> >Here's my kafka_metrics.json
> >> >{
> >> >  "servers" : [ {
> >> >    "port" : "9999",
> >> >    "host" : "ecokaf1",
> >> >    "queries" : [ {
> >> >      "outputWriters" : [ {
> >> >        "@class" :
> >> >"com.googlecode.jmxtrans.model.output.GangliaWriter",
> >> >        "settings" : {
> >> >          "groupName" : "kafka stats 2",
> >> >          "port" : 8649,
> >> >          "host" : "ecokaf1"
> >> >        }
> >> >      } ],
> >> >      "obj" : "kafka.server:type=ReplicaManager,name=PartitionCount",
> >> >      "resultAlias": "Kafka.ReplicaManager",
> >> >      "attr" : [ "Value" ]
> >> >    } ],
> >> >    "numQueryThreads" : 2
> >> >  } ]
> >> >}
> >> >
> >> >I start jmxtrans as follows:
> >> >sudo ./jmxtrans.sh start kafka_metrics.json
> >> >
> >> >
> >> >JMXTrans starts without any issues. Here are the logs:
> >> >
> >> >[05 Nov 2013 00:49:48] [main] 0      INFO
> >> >(com.googlecode.jmxtrans.JmxTransformer:134) - Starting Jmxtrans on :

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB