Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - JMXTrans not sending kafka 0.8 metrics to Ganglia


+
Priya Matpadi 2013-11-05, 01:02
+
Paul Mackles 2013-11-05, 01:11
+
Priya Matpadi 2013-11-05, 06:00
+
Paul Mackles 2013-11-05, 20:36
Copy link to this message
-
Re: JMXTrans not sending kafka 0.8 metrics to Ganglia
Priya Matpadi 2013-11-06, 18:07
typeNames is required in settings section for beanName to be included in
the metric name when jmxtrans sends it out. Here's an example:

      "outputWriters" : [ {
        "@class" :
"com.googlecode.jmxtrans.model.output.GangliaWriter",
        "settings" : {
          "groupName" : "kafka",
          "port" : 8649,
          "host" : "hostname",
          "typeNames" : [ "name" ]
        }
      } ],
      "obj": "\"kafka.server\":type=\"BrokerTopicMetrics\",name=\"*\"",
      "attr": [
      "Count",
      "FifteenMinuteRate",
      "FiveMinuteRate",
      "OneMinuteRate",
      "MeanRate"
      ],
      "resultAlias": "BrokerTopicMetrics"

    } ]
On Tue, Nov 5, 2013 at 12:35 PM, Paul Mackles <[EMAIL PROTECTED]> wrote:

> In our case, jmxtrans returns:
>
> ReplicaManager.ISRShrinksPerSec.Count
> ReplicaManager.ISRShrinksPerSec.MeanRate
>
> ReplicaManager.ISRExpandsPerSec.OneMinuteRate
>
> ReplicaManager.ISRExpandsPerSec.Count
> ReplicaManager.ISRExpandsPerSec.MeanRate
> ReplicaManager.ISRExpandsPerSec.OneMinuteRate
> ReplicaManager.LeaderCount.Value
>
> ReplicaManager.PartitionCount.Value
>
>
>
> In other words, the bean name is include in the metric name when jmxtrans
> sends it out. In our case, we are using graphite. Not sure if that makes a
> difference.
>
> Paul
>
> On 11/5/13 12:59 AM, "Priya Matpadi" <[EMAIL PROTECTED]> wrote:
>
> >Paul,
> >Thank you! I didn't realize the quotes are important to keep.
> >I have question on name=\"*\""
> >
> >For the type=ReplicaManager, there are at least 4 Mbean names
> >"kafka.server":name="PartitionCount",type="ReplicaManager"
> >"kafka.server":name="LeaderCount",type="ReplicaManager"
> >
> >"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"
> >"kafka.server":name="ISRExpandsPerSec",type="ReplicaManager"
> >
> >In ganglia metrics dropdown, I only see 4 elements as configured above
> >i.e.
> >
> >ReplicaManager.Count
> >ReplicaManager.OneMinuteRate
> >ReplicaManager.MeanRate
> >ReplicaManager.Value
> >
> >Then, how can one tell if for e.g. Value pertains to LeaderCount or
> >PartitionCount?
> >Even though it is laborious, isn't it good idea to explicitly specify the
> >name?
> >
> >Thanks,
> >Priya
> >
> >
> >
> >
> >On Mon, Nov 4, 2013 at 5:10 PM, Paul Mackles <[EMAIL PROTECTED]> wrote:
> >
> >> It looks like you are missing quotes in the object name. Here is a
> >>snippet
> >> from our jmxtrans configs:
> >>
> >> "resultAlias": "ReplicaManager",
> >>      "obj": "\"kafka.server\":type=\"ReplicaManager\",name=\"*\"",
> >>      "attr": [
> >>       "Count",
> >>       "OneMinuteRate",
> >>       "MeanRate",
> >>       "Value"
> >>      ]
> >>
> >>
> >> Unless more recent versions of kafka get rid of the quotes (we are on an
> >> older 0.8 version still).
> >>
> >> Paul
> >>
> >> On 11/4/13 8:02 PM, "Priya Matpadi" <[EMAIL PROTECTED]>
> wrote:
> >>
> >> >I am trying to send kafka metrics for display to ganglia server using
> >> >latest download from https://github.com/adambarthelson/kafka-ganglia.
> >> >
> >> >Here's my kafka_metrics.json
> >> >{
> >> >  "servers" : [ {
> >> >    "port" : "9999",
> >> >    "host" : "ecokaf1",
> >> >    "queries" : [ {
> >> >      "outputWriters" : [ {
> >> >        "@class" :
> >> >"com.googlecode.jmxtrans.model.output.GangliaWriter",
> >> >        "settings" : {
> >> >          "groupName" : "kafka stats 2",
> >> >          "port" : 8649,
> >> >          "host" : "ecokaf1"
> >> >        }
> >> >      } ],
> >> >      "obj" : "kafka.server:type=ReplicaManager,name=PartitionCount",
> >> >      "resultAlias": "Kafka.ReplicaManager",
> >> >      "attr" : [ "Value" ]
> >> >    } ],
> >> >    "numQueryThreads" : 2
> >> >  } ]
> >> >}
> >> >
> >> >I start jmxtrans as follows:
> >> >sudo ./jmxtrans.sh start kafka_metrics.json
> >> >
> >> >
> >> >JMXTrans starts without any issues. Here are the logs:
> >> >
> >> >[05 Nov 2013 00:49:48] [main] 0      INFO
> >> >(com.googlecode.jmxtrans.JmxTransformer:134) - Starting Jmxtrans on :