Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> JMXTrans not sending kafka 0.8 metrics to Ganglia


Copy link to this message
-
JMXTrans not sending kafka 0.8 metrics to Ganglia
I am trying to send kafka metrics for display to ganglia server using
latest download from https://github.com/adambarthelson/kafka-ganglia.

Here's my kafka_metrics.json
{
  "servers" : [ {
    "port" : "9999",
    "host" : "ecokaf1",
    "queries" : [ {
      "outputWriters" : [ {
        "@class" :
"com.googlecode.jmxtrans.model.output.GangliaWriter",
        "settings" : {
          "groupName" : "kafka stats 2",
          "port" : 8649,
          "host" : "ecokaf1"
        }
      } ],
      "obj" : "kafka.server:type=ReplicaManager,name=PartitionCount",
      "resultAlias": "Kafka.ReplicaManager",
      "attr" : [ "Value" ]
    } ],
    "numQueryThreads" : 2
  } ]
}

I start jmxtrans as follows:
sudo ./jmxtrans.sh start kafka_metrics.json
JMXTrans starts without any issues. Here are the logs:

[05 Nov 2013 00:49:48] [main] 0      INFO
(com.googlecode.jmxtrans.JmxTransformer:134) - Starting Jmxtrans on :
kafka_metrics.json
[05 Nov 2013 00:49:48] [main] 386    DEBUG
(com.googlecode.jmxtrans.JmxTransformer:354) - Loaded file:
/usr/share/jmxtrans/kafka_metrics.json
[05 Nov 2013 00:49:48] [main] 392    DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:119) - Validated
Ganglia metric [host: ecokaf1, port: 8649, addressingMode: UNICAST, ttl: 5,
v3.1: true, units: '', slope: BOTH, tmax: 60, dmax: 0, spoofedHostName:
192.168.3.1:ecokaf1, groupName: 'kafka stats 2']
[05 Nov 2013 00:49:48] [main] 406    DEBUG
(com.googlecode.jmxtrans.JmxTransformer:429) - Scheduled job:
ecokaf1:9999-1383612588479-3580845919 for server: Server [host=ecokaf1,
port=9999, url=null, cronExpression=null, numQueryThreads=2]
[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 414    DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
Server [host=ecokaf1, port=9999, url=null, cronExpression=null,
numQueryThreads=2]
[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 544    DEBUG
(com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 1 query threads
[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 553    DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job:
Server [host=ecokaf1, port=9999,
url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi, cronExpression=null,
numQueryThreads=2]
[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60401  DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
Server [host=ecokaf1, port=9999,
url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi, cronExpression=null,
numQueryThreads=2]
[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60404  DEBUG
(com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 1 query threads
[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60410  DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job:
Server [host=ecokaf1, port=9999,
url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi, cronExpression=null,
numQueryThreads=2]

Does any one spot an issue with the json?

When I add the jvmheapmemory and jvmGC examples to my json, jmxtrans sends
those metrics to Ganglia. Corresponding rrds files are created in ganglia
location and I can see the following in jmxtrans.log:
[05 Nov 2013 00:45:44] [pool-30-thread-1] 1740436 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending Ganglia
metric heap.HeapMemoryUsage_init=263441792
[05 Nov 2013 00:45:44] [pool-30-thread-2] 1740449 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending Ganglia
metric GC.PSScavenge.CollectionCount=334

And I know that kafka is publishing jmx metrics because I can see them
using jmxsh-R5 command line tool,

Appreciate any pointers you can provide with regards to kafka.
Thanks,
Priya