Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> JMXTrans not sending kafka 0.8 metrics to Ganglia


Copy link to this message
-
JMXTrans not sending kafka 0.8 metrics to Ganglia
I am trying to send kafka metrics for display to ganglia server using
latest download from https://github.com/adambarthelson/kafka-ganglia.

Here's my kafka_metrics.json
{
  "servers" : [ {
    "port" : "9999",
    "host" : "ecokaf1",
    "queries" : [ {
      "outputWriters" : [ {
        "@class" :
"com.googlecode.jmxtrans.model.output.GangliaWriter",
        "settings" : {
          "groupName" : "kafka stats 2",
          "port" : 8649,
          "host" : "ecokaf1"
        }
      } ],
      "obj" : "kafka.server:type=ReplicaManager,name=PartitionCount",
      "resultAlias": "Kafka.ReplicaManager",
      "attr" : [ "Value" ]
    } ],
    "numQueryThreads" : 2
  } ]
}

I start jmxtrans as follows:
sudo ./jmxtrans.sh start kafka_metrics.json
JMXTrans starts without any issues. Here are the logs:

[05 Nov 2013 00:49:48] [main] 0      INFO
(com.googlecode.jmxtrans.JmxTransformer:134) - Starting Jmxtrans on :
kafka_metrics.json
[05 Nov 2013 00:49:48] [main] 386    DEBUG
(com.googlecode.jmxtrans.JmxTransformer:354) - Loaded file:
/usr/share/jmxtrans/kafka_metrics.json
[05 Nov 2013 00:49:48] [main] 392    DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:119) - Validated
Ganglia metric [host: ecokaf1, port: 8649, addressingMode: UNICAST, ttl: 5,
v3.1: true, units: '', slope: BOTH, tmax: 60, dmax: 0, spoofedHostName:
192.168.3.1:ecokaf1, groupName: 'kafka stats 2']
[05 Nov 2013 00:49:48] [main] 406    DEBUG
(com.googlecode.jmxtrans.JmxTransformer:429) - Scheduled job:
ecokaf1:9999-1383612588479-3580845919 for server: Server [host=ecokaf1,
port=9999, url=null, cronExpression=null, numQueryThreads=2]
[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 414    DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
Server [host=ecokaf1, port=9999, url=null, cronExpression=null,
numQueryThreads=2]
[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 544    DEBUG
(com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 1 query threads
[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 553    DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job:
Server [host=ecokaf1, port=9999,
url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi, cronExpression=null,
numQueryThreads=2]
[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60401  DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
Server [host=ecokaf1, port=9999,
url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi, cronExpression=null,
numQueryThreads=2]
[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60404  DEBUG
(com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 1 query threads
[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60410  DEBUG
(com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job:
Server [host=ecokaf1, port=9999,
url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi, cronExpression=null,
numQueryThreads=2]

Does any one spot an issue with the json?

When I add the jvmheapmemory and jvmGC examples to my json, jmxtrans sends
those metrics to Ganglia. Corresponding rrds files are created in ganglia
location and I can see the following in jmxtrans.log:
[05 Nov 2013 00:45:44] [pool-30-thread-1] 1740436 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending Ganglia
metric heap.HeapMemoryUsage_init=263441792
[05 Nov 2013 00:45:44] [pool-30-thread-2] 1740449 DEBUG
(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending Ganglia
metric GC.PSScavenge.CollectionCount=334

And I know that kafka is publishing jmx metrics because I can see them
using jmxsh-R5 command line tool,

Appreciate any pointers you can provide with regards to kafka.
Thanks,
Priya

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB